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Weapon Systems Annual Assessment 

Limited Use of Knowledge-Based Practices Continues to 
Undercut DOD’s Investments 


This special report is GAO’s 17th annual assessment of the Department of 
Defense’s (DOD) $1.69 trillion portfolio of 82 major weapon systems acquisition 
programs. This report (1) examines changes in the portfolio since last year, (2) 
evaluates DOD’s actions in areas related to recent, selected acquisition reforms, 
and (3) offers a quick look at cost and schedule performance of 51 individual 
weapon programs based on DOD documentation and questionnaire responses 
from their respective program offices. 


For years, GAO has highlighted the importance of applying knowledge-based 
acquisition practices as a way to improve DOD’s program outcomes. When 
programs enter development with insufficient knowledge, negative effects often 
cascade throughout the acquisition cycle. The figure below shows how three 
acquisition phases—technology development, system development, and 
production—align with three key points for demonstrating knowledge. 
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DOD’s 2018 portfolio of major weapon programs has grown in cost by $8 billion, 
but contains four fewer systems than last year. GAO’s analysis suggests that 
one of the primary drivers of this cost growth is that, since 2012, the average age 
of programs has increased—indicative of DOD decisions to introduce new 
capabilities through additions to existing programs rather than by starting new 
programs. On average, programs in the current portfolio are about 4 months 
older than last year and nearly 3 years older than in 2012. (See figure.) 
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Portfolio-wide cost growth has occurred in an environment where awards are 
often made without full and open competition. Specifically, GAO found that DOD 
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did not compete 67 percent of 183 major contracts currently reported for its 82 
major programs. GAO also observed that DOD awarded 47 percent of these 183 
contracts to five corporations and entities connected with them. (See figure.) 

DOD Programs Competed One-Third of Currently Reported Major Contracts 
with Nearly Half of Awards Concentrated within Five Companies 
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Although Knowledge-Based 
Acquisition Practices Can Lead 
to Better Cost and Schedule 
Outcomes, Programs Continue 
Not to Fully Implement Them 


DOD Major Defense Acquisition Programs Continue Not to Fully Implement 
Key Knowledge-Based Acquisition Practices 
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This lack of knowledge and the effects it can have throughout a program’s 
acquisition life cycle can increase the risk of undesirable cost and schedule 
outcomes. In 2018, GAO conducted an exploratory statistical analysis of 15 
programs in production to evaluate their completion of the eight key knowledge- 
based acquisition practices. In 2019, GAO expanded this analysis to include 17 
programs. Over the past two years, GAO found that the major DOD acquisition 
programs that completed one or more of three specific practices had significantly 
lower cost and schedule growth than those that did not. These three practices 
were (1) demonstration that all critical technologies were very close to final form, 
fit, and function, within a relevant environment, before starting development; (2) 
completion of a preliminary design review prior to starting development; and (3) 
release of at least 90 percent of design drawings by critical design review. 


As we have reported in previous assessments, DOD programs continue not to 
fully implement knowledge-based acquisition practices. GAO observed that most 
of the 45 current programs proceeded into system development, through critical 
design reviews, and into production without completing the key knowledge-based 
practices associated with each of these three points. (See table.) 
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U.S. GOVERNMENT ACCOUNTABILITY OFFICE 


441 G St. N.W. 
Washington, DC 20548 


Comptroller General 
of the United States 


May 7, 2019 

Congressional Committees 

I am pleased to present our 17th annual assessment of the Department of 
Defense’s (DOD) major weapon system acquisition programs—an area 
on GAO’s high-risk list since 1990. 1 This year’s report offers observations 
on the performance of DOD’s 2018 portfolio of 82 major programs, which 
the department expects to cost $1.69 trillion in total. 2 This significant 
financial investment demands keen oversight and continued 
implementation of legislative reforms and policies aimed at improving 
DOD’s stewardship. These include use of knowledge-based acquisition 
practices, which we have previously recommended to DOD, but which 
continue to lack consistent application within the department. 

This year’s assessment comes at a time of significant change within the 
department. In response to congressional direction, in February 2018, 
DOD dissolved the Office of the Under Secretary of Defense for 
Acquisition, Technology and Logistics. DOD has developed a new 
organizational structure that refocuses the Office of the Secretary of 
Defense’s (OSD) principal acquisition function to one focused on 
increasing the speed of capability development rather than conducting 
program oversight. As part of this restructure, overall responsibilities for 
program oversight, which DOD calls “milestone decision authority,” now 
generally reside within the three military departments. 

We recognize the magnitude of these changes and, through our ongoing 
work, know that OSD and the military departments have made some 


^AO, High-Risk Series: Substantial Efforts Needed to Achieve Greater Progress on 
High-Risk Areas, GAO-19-157SP (Washington, D.C.: Mar. 6, 2019). 

2 Our assessment of DOD’s portfolio does not include the cost of the Ballistic Missile 
Defense System (BMDS) as the program and its elements lack acquisition program 
baselines needed to support our assessment of cost and schedule change. Although 10 
U.S.C. § 225 requires the Missile Defense Agency (MDA) to establish and maintain an 
acquisition baseline for certain elements of the BMDS, these baselines are not the same 
as the acquisition program baselines developed pursuant to 10 U.S.C. § 2435 and DOD 
acquisition policies, and they do not provide all the data we need to assess cost and 
schedule changes. For more information on BMDS and its elements, see GAO, Missile 
Defense: The Warfighter and Decision Makers Would Benefit from Better Communication 
about the System’s Capabilities and Limitations, GAO-18-324 (Washington, D.C.: May 30, 
2018). 
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progress implementing them. At the same time, we observed this year 
that cost performance in major programs continues to slip. Most troubling 
is that DOD’s newer programs—those initiated after implementation of 
major acquisition reforms in 2010—now show aggregate cost growth. In 
previous assessments, we observed that this group of programs 
generated cost decreases that, in part, offset cost increases within the 
rest of the portfolio. 

We attribute the deteriorating performance of newer programs to the 
inconsistent implementation of knowledge-based acquisition practices—a 
condition we highlighted in our 2018 assessment. Our work has found 
that when programs enter development with insufficient knowledge, 
negative effects can cascade throughout the acquisition cycle. These 
knowledge shortfalls, or gaps, often begin with program decisions to 
accept immature technologies at the start of system development, but 
then later manifest in other forms as the program approaches production. 

In this environment, decision makers are confronted with the choice of 
increasing program investments, despite lacking visibility on whether the 
program’s cost and schedule estimates are achievable, or truncating the 
program and subsequently depriving warfighters of a needed capability. 
We have made numerous recommendations over the years to address 
these knowledge gaps in DOD’s programs. While DOD has often agreed 
with these recommendations, and incorporated them into its acquisition 
policy, it has inconsistently applied the policy to its acquisition programs. 

Further, for the second year in a row, we completed an exploratory 
statistical analysis that continues to validate a linkage between the 
attainment of knowledge and the real-life cost and schedule outcomes 
that programs deliver. This year we expanded our data set by two 
programs to a total of 17, each of which has entered production. Over the 
past two years, our analyses show that, consistent with our best 
practices, programs that attained certain knowledge at key points had 
lower cost and schedule growth than other programs. 

As evidence of the importance of knowledge-based acquisition practices 
continues to mount, we are troubled by DOD’s continued reluctance to 
fully adopt them. Service acquisition executives, now assigned as the 
milestone decision authority for most major programs within their military 
departments, are positioned to insist that programs under their purview 
implement knowledge-based practices. Such leadership, if displayed, 
could help reverse the cost growth we observed and position the military 
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departments to obtain persistent efficiencies in the acquisition programs 
they manage. 



Gene L. Dodaro 

Comptroller General of the United States 
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GAO 

441 G St. N.W. 


U.S. GOVERNMENT ACCOUNTABILITY OFFICE 


Washington, DC 20548 


May 7, 2019 

Congressional Committees 

In response to the joint explanatory statement accompanying the 
Department of Defense Appropriations Act, 2009, this report provides 
insight into the department’s $1.69 trillion portfolio of major weapon 
programs. 1 It includes observations on (1) the cost and schedule 
performance of DOD’s 2018 portfolio of 82 major weapon programs, (2) 
the extent to which 51 current and future programs (i.e., those that have 
not yet begun development) are implementing recent key acquisition 
reform initiatives, and (3) the knowledge attained by 51 current and future 
programs attained at key decision points in the acquisition process. This 
report also includes information related to small business participation, 
pursuant to a provision in a report to the National Defense Authorization 
Act (NDAA) for Fiscal Year 2013. 2 Specifically, we determined whether 
individual subcontracting reports from a program’s prime contractor or 
contractors were accepted within the Electronic Subcontracting Reporting 
System (eSRS). 3 Results from this analysis can be found in appendix I. 

Our observations in this report are based on three sets of programs: 


^ee Explanatory Statement, 154 Cong. Rec. H 9427, 9526 (daily ed., Sept. 24, 2008), to 
the Department of Defense Appropriations Act, 2009, contained in Division C of the 
Consolidated Security, Disaster Assistance, and Continuing Appropriations Act, 2009, 
Pub. L. No. 110-329 (2008). 

2 H.R. Rep. No. 112-479, at 284 (2012). The National Defense Authorization Act for Fiscal 
Year 2013, Pub. L. No. 112-239. 

3 The government uses individual subcontracting reports on eSRS as one method of 
monitoring small business participation, as the report includes goals for small business 
subcontracting. 
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• We assessed 82 major defense acquisition programs (MDAP) in 
DOD’s 2018 portfolio for cost and schedule performance and their use 
of competition in contracting. 4 

• First, we obtained cost, schedule, and quantity data from DOD’s 
December 2017 Selected Acquisition Reports (SAR)—which detail 
initial cost, schedule, and performance baselines and changes 
over the past year—and from the Defense Acquisition 
Management Information Retrieval (DAMIR) system, a DOD 
repository for program data. We assessed data reliability by 
comparing the SAR data we entered into our weapon system 
database and the DAMIR data and determined the data were 
sufficiently reliable for the purposes of our report. 

• Next, we identified active major development and procurement 
contracts that programs reported in their December 2017 SARs 
and entered those contract numbers into the Federal Procurement 
Data System—Next Generation (FPDS-NG) to obtain data specific 
to those contracts. 5 We assessed data reliability by obtaining and 
reviewing a federal-wide summary of the system’s completeness 
and accuracy and the data validation rules and determined that 
the data were sufficiently reliable for the purposes of our report. 

• We also assessed 45 MDAPs currently between the start of 
development and the early stages of production. We developed a 
questionnaire to obtain information on the extent to which these 
programs are following knowledge-based acquisition practices for 
technology maturity, design stability, and production readiness. The 
questionnaire asked program officials to provide information about 
systems engineering, design drawings, manufacturing planning and 
execution, and the implementation of specific acquisition reforms. In 
addition, the questionnaire requested that program officials provide 


4 Major defense acquisition programs (MDAP) are those identified by DOD or that have a 
dollar value for all increments estimated to require eventual total expenditure for research, 
development, test, and evaluation of more than $480 million, or for procurement of more 
than $2.79 billion, in fiscal year 2014 constant dollars. DOD maintains a list of programs 
designated as future MDAPs. These programs have not formally been designated as 
MDAPs; however, DOD plans for these programs to enter system development, or bypass 
development and begin production, at which point DOD will likely designate them as 
MDAPs. We refer to these programs as future or pre-MDAPs throughout this report. 
Recent amendments to the statutory definition of an MDAP expressly exclude those 
acquisitions using the rapid fielding or rapid prototyping acquisition pathway described in 
section 804 of the National Defense Authorization Act for Fiscal Year 2016. 

5 FPDS-NG is an automated system used to collect and report on federal procurement 
spending and the authoritative source for government-wide contract award data. 


Page 5 


GAO-19-336SP Weapon Systems Annual Assessment 



details on scheduling, critical technology levels, major development 
and early procurement contract data, and other information. We 
received questionnaire responses from all 45 current MDAPs from 
October 2018 through January 2019. 

• We also assessed six future MDAPs not yet in the portfolio to gain 
additional insights into knowledge they plan to attain before starting 
development and their plans for implementing recent key acquisition 
reforms. We provided a questionnaire to program offices to obtain 
information on schedule events, costs, and acquisition reforms, and 
received responses from all six future programs from October 2018 
through December 2018. 


In addition, we present individual assessments of 51 MDAPs, which 
include the 45 MDAPs currently in development or early production, as 
well as the six future programs. We used December 2017 SAR data as a 
starting point for reporting program cost information in the individual 
assessments. For most assessments, we were able to include more 
current cost information that we obtained through our web-based 
questionnaires. Appendix II provides additional information on our 
objectives, scope, and methodology. 

We conducted this performance audit from May 2018 to May 2019 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 


Background 


DOD acquires new weapons for its warfighters through a management 
process known as the Defense Acquisition System. 6 This system includes 
three phases that defense acquisition programs generally proceed 
through, which are (1) technology maturation and risk reduction, (2) 
engineering and manufacturing development, and (3) production and 
deployment. In this report, we refer to these three phases more simply as 


department of Defense Directive 5000.01, The Defense Acquisition System (May 2003 
[incorporating change 2 (Aug. 2018)]); Department of Defense Instruction 5000.02, 
Operation of the Defense Acquisition System (Jan. 2015) [incorporating change 4 (Aug. 
2018)] (“DOD Instruction 5000.02”). 
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technology development, system development, and production. Programs 
typically complete a series of milestone reviews and other key decision 
points that authorize entry into a new acquisition phase. 

Our body of work has shown that attaining high levels of knowledge 
before programs make significant commitments during product 
development drives positive acquisition outcomes. 7 We have found that in 
order to reduce risk there are three key points where programs should 
demonstrate critical levels of knowledge before proceeding to the next 
acquisition phase: development start, system-level critical design review, 
and production start. Figure 1 aligns the acquisition milestones described 
in DOD Instruction 5000.02, which establishes policy for the management 
of acquisition programs, with these three key decision points. 


7 GAO, Best Practices: DOD Can Achieve Better Outcomes by Standardizing the Way 
Manufacturing Risks Are Managed, GAO-10-439 (Washington, D.C.: Apr. 22, 2010); Best 
Practices: High Levels of Knowledge at Key Points Differentiate Commercial Shipbuilding 
from Navy Shipbuilding, GAO-09-322 (Washington, D.C.: May 13, 2009); Defense 
Acquisitions: A Knowledge-Based Funding Approach Could Improve Major Weapon 
System Program Outcomes, GAO-08-619 (Washington, D.C.: July 2, 2008); Best 
Practices: Capturing Design and Manufacturing Knowledge Early Improves Acquisition 
Outcomes, GAO-02-701 (Washington, D.C.: July 15, 2002); Best Practices: Better 
Matching of Needs and Resources Will Lead to Better Weapon System Outcomes, 
GAO-01-288 (Washington, D.C.: Mar. 8, 2001); and Best Practices: Better Management of 
Technology Development Can Improve Weapon System Outcomes, GAO/NSIAD-99-162 
(Washington, D.C.: July 30, 1999). 
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Figure 1: DOD Acquisition Process and GAO-identified Knowledge Points 

Department of Defense (DOD) acquisition process: 
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Source: GAO analysis of DOD-provided data, DOD Instruction 5000.02, and best practices. | GAO-19-336SP 


Knowledge associated with these three points builds over time. Our prior 
work on knowledge-based approaches shows that a knowledge deficit 
early in a program can cascade through design and production, leaving 
decision makers with less knowledge to support decisions about when 
and how to move into subsequent acquisition phases that require more 
budgetary resources. Under a knowledge-based approach, demonstrating 
technology maturity is a prerequisite for moving forward into system 
development, during which time the focus should be on design and 
integration. Similarly, a stable and mature design is a prerequisite for 
moving into production, where the focus should be on efficient 
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manufacturing. Appendix III provides additional details about key 
practices at each of the knowledge points. To accommodate shipbuilding 
programs in this report, we correlated detail design contract awards, 
fabrication starts, and lead ship deliveries with development start, critical 
design review, and production start, respectively. 

Our work has led to multiple recommendations that DOD has generally or 
partially agreed with and took steps to implement some of them in its 
acquisition policy. For example, our previous work recommended DOD 
ensure programs conduct a preliminary design review prior to starting 
development, and DOD’s policy now reflects this. Further, our work has 
influenced efforts within DOD and Congress to address some of the 
challenges in the defense acquisition system—primarily, that it takes 
longer and costs more to develop and produce the systems required to 
perform DOD’s various missions and operations. Notably, the Weapon 
Systems Acquisition Reform Act of 2009 (WSARA) sought to improve the 
way DOD acquires major weapon systems and incorporated many of our 
related recommendations. 8 Our work has shown that DOD, through its 
Instruction 5000.02, has incorporated WSARA and other initiatives to 
address sound management practices, such as realistic cost estimates, 
use of prototypes, and systems engineering. 

More recently, the Congress has passed legislation that introduced 
numerous additional acquisition reforms related to improving the 
performance of weapon acquisition programs. For this assessment, we 
reviewed MDAP implementation of several of these reforms. We focused 
on the reforms that involve oversight and acquisition principles consistent 
with our knowledge-based practices and provide mechanisms for more 
rapid fielding of warfighter capability. We explain the reforms in greater 
detail in appendix IV. These acquisition reforms are summarized as 
follows: 

• The NDAA for fiscal year 2016 contained specific provisions that 
authorize changes in certain areas of DOD’s acquisition process. 9 For 
example, some of these provisions included language pertaining to 
the alternate pathways for rapidly prototyping and fielding new 
capabilities, waivers of certain acquisition-related laws, and revisions 
to milestone decision responsibilities for MDAPs. The NDAA for fiscal 


8 Pub. L. No. 111-23. 

9 Pub. L. No. 114-92. 
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year 2016 also contained a provision that permanently codified DOD’s 
“other transaction” authority for prototype projects. Under this 
authority, programs can enter into what are commonly referred to as 
“other transaction agreements,” which generally are not subject to 
federal acquisition or assistance statutes and regulations. 
Consequently, this authority provides agencies more flexibility to meet 
project requirements and mission needs. 

• The NDAA for fiscal year 2017 included provisions that require new 
acquisition programs to be designed and developed, to the maximum 
extent practicable, with a modular, open systems approach to enable 
incremental development and enhance competition, innovation, and 
interoperability. 10 It also included provisions that require an 
independent technical risk assessment in certain instances and goals 
for procurement unit cost, sustainment costs, initial operational 
capability, technology maturation, and prototyping. 

• The NDAA for fiscal year 2018 contained a provision titled 
“expanded other transaction authority for prototype projects” that 
increased the dollar values of the DOD-internal approval thresholds. 


The 2016 and 2017 NDAAs also contained provisions that altered the 

roles and responsibilities for MDAP oversight to give significantly more 

authority to the military departments for managing acquisition programs. 

Among other things, these provisions required the following: 

• The service acquisition executive of the military department 
concerned be designated as the milestone decision authority for most 
MDAPs initiated after October 1,2016; 

• A process to address requests from service acquisition executives to 
revert existing programs’ milestone decision authority to them; 

• The establishment of two new executive offices—the Offices of the 
Under Secretary of Defense for Research and Engineering and the 
Under Secretary of Defense for Acquisition and Sustainment, which 
replaced the Under Secretary of Defense for Acquisition, Technology 
and Logistics; and 

• Use of an alternate milestone decision authority, under certain 
circumstances. 


10 Pub. L. No. 114-328. 
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The movement of milestone decision authority has begun to take place, 
as shown in figure 2. 


Figure 2: Milestone Decision Authority in DOD’s Portfolio of Major Defense 
Acquisition Programs, 2012-2019 


Number of major Defense Acquisition programs 
100 



2012 2013 2014 2015 2016 2017 2018 March 2019 



Office of the Secretary of Defense 
Military department 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 

Note: The figure does not account for classified programs, which we excluded from our analyses. 


We have ongoing work to more comprehensively assess DOD’s 
implementation of these and other recent acquisition reforms and we 
expect to issue a report in spring 2019. As part of that review, we are 
evaluating DOD’s new organizational structure that refocuses the Office 
of the Secretary of Defense’s principal role from program oversight to one 
intended to ensure that major DOD investments produce integrated, 
technically superior capability that consistently outpaces global threats. 
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Nine Observations on 
the Performance of 
DOD’s 2018 MDAP 
Portfolio 


DOD’s 2018 MDAP portfolio consists of 82 programs and will cost $1.69 
trillion to acquire. 11 We identified MDAPs as being in the 2018 portfolio if 
they published a December 2017 SAR. The portfolio is smaller yet more 
expensive than last year’s portfolio. The estimated total acquisition cost of 
the portfolio has grown by 51 percent, or $569 billion, since the identified 
programs’ first full estimates. While programs initiated since 2010 
continue to outperform those initiated earlier, our analysis of the current 
portfolio indicates that this trend has weakened. The result is portfolio¬ 
wide cost growth and a loss of buying power gains. Moreover, this growth 
has occurred in an acquisition environment where DOD has awarded 86 
of the 183 major contracts that we reviewed to five corporations or entities 
connected with them and where most of the major contracts as reported 
in the SARs were awarded without full and open competition or on a sole 
source basis. In addition, current MDAPs did not compete 123 of these 
183 contracts. 


DOD’s 2018 Portfolio 
Invested More Money 
across Fewer Programs 


Observation 1. The 2018 portfolio consists of 82 programs that will 
cost $1.69 trillion to acquire. This portfolio has four fewer programs 
than last year’s portfolio, but will require more money than any 
portfolio from the prior six years. 

• The current portfolio is smaller yet more expensive that last year’s 
portfolio. This is the third time in 10 years that a decrease in the 
number of portfolio systems was not accompanied by a corresponding 
decrease in expected costs. Specifically, the 2018 portfolio is 14 
percent smaller than the 2008 portfolio, yet its projected completion 
cost is only 3 percent less since that time. 

• The average total acquisition cost per program has continued to 
increase over the last decade. In 2008, the average cost of an 
acquisition program was $18.3 billion, while the comparative average 
cost for 2018 is $20.6 billion. 


Figure 3 shows the number of programs and the total cost of the 2018 
portfolio in relation to previous years. 


11 All dollar figures are in fiscal year 2019 constant dollars, unless otherwise noted. 
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Figure 3: Historical DOD Portfolio Comparison 

Fiscal year 2019 dollars (in billions) Number of programs 



Portfolio year 

Total acquisition cost 
Number of programs 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 

Note: DOD did not issue Selected Acquisition Reports (SAR) in 2009, which prevents us from 
obtaining the cost baseline information necessary to include 2009 in this analysis. 


• In 2018, two programs entered the portfolio and six exited. DOD 
approved the two entering programs for system development starts 
between January and December 2018. For five of the six exited 
programs, 90 percent of planned expenditures have been made. The 
Army terminated the remaining program, Warfighter Information 
Network—Tactical (WIN-T) Increment 3, due to affordability 
challenges and changes to the Army’s network strategy. 

Figure 4 shows programs entering and exiting the 2018 portfolio. 
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Figure 4: Programs that Entered and Exited the 2018 Portfolio 
ENTERING PROGRAMS 



EXITING PROGRAMS 

Programs completed life cycle 



MH-60R 




GBS 


Program terminated 


WIN-T 
Inc 3 


CH-47F Block II 

T-AO 205 Class 

C-5 RERP 

MH-60R 

WIN-T Inc 3 

GBS 

EPS 

WGS 


CH-47F Modernized Cargo Helicopter 

T-AO 205 John Lewis Class Fleet Replenishment Oiler 

C-5 Reliability Enhancement and Re-engineering Program 

MH-60R Multi-Mission Helicopter 

Warfighter Information Network-Tactical Increment 3 

Global Broadcast Service 

Enhanced Polar System 

Wideband Global SATCOM 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 

Note: The two entering programs produced their first Selected Acquisition Reports in December 2017, 
while the six exiting programs produced their final Selected Acquisition Reports in December 2016. 
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Observation 2. The average age of programs in the portfolio has 
increased. 

While the number of programs in the portfolio has declined since last 
year, the average age of the programs that have remained has steadily 
increased since 2012. The current portfolio is approximately 4 months 
older than last year’s and nearly 3 years older than that from 2012. Figure 
5 displays the increasing average age of programs in the portfolio since 
2012 . 


Figure 5: Annual Change in Average Program Age in DOD’s Portfolio of Major 
Defense Acquisition Programs, 2012-2018 

Program age (in years) 

16 



8 

6 

4 

2 

0 

2012 2013 2014 2015 2016 2017 2018 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


The growing portfolio age is a byproduct of multiple factors, including the 
following: 

• DOD has introduced new capabilities and upgrades through additions 
to existing programs rather than by starting new programs—an 
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approach inconsistent with best practices. 12 Best practices 
recommend an incremental approach in which new development 
efforts are structured and managed as separate acquisition programs 
with their own requirements and acquisition program baselines. If the 
effort is not established as a separate acquisition program, 
transparency is limited and it may be more difficult for Congress to 
hold the new effort accountable for achieving its cost, schedule, and 
performance requirements. MDAPs to which DOD has added 
warfighting capabilities under the existing program structure include 
the DOD-wide F-35 Lightning II Joint Strike Fighter and the Navy’s P- 
8A, Arleigh Burke Class Destroyer (DDG 51), and Virginia Class 
Submarine (SSN 774). 

• Twenty programs have remained in the DOD MDAP portfolio for more 
than 20 years. Programs using DOD’s middle tier rapid prototyping 
and rapid fielding pathways, established pursuant to Section 804 of 
the fiscal year 2016 NDAA, are expressly excluded from the statutory 
definition of MDAP and therefore will not join the portfolio. For 
example, during the course of our review, six programs that DOD 
previously classified as pre-MDAPs are now proceeding via the 
alternate rapid prototyping and rapid fielding acquisition pathways. As 
newer programs choose this alternate path, the portfolio takes in 
fewer new programs than it otherwise would, which skews the 
average age of the portfolio toward older programs. 


Observation 3. Future funding needs remain within historic norms. 

• The current amount of future funding the portfolio needs decreased 
from $701.8 billion to $688.8 billion over the past year. 13 This 
decrease is largely due to a nearly $7 billion decrease in development 
and procurement costs since 2017 in the Army’s WIN-T Increment 2 
program, which the Army has descoped. For the 2018 portfolio, DOD 
has programmed $32.6 billion for development and $656.2 billion for 
procurement activities. 


12 GAO, F-35 Joint Strike Fighter: Continued Oversight Needed as Program Plans to Begin 
Development of New Capabilities, GAO-16-390 (Washington, D.C.: Apr. 14, 2016); GAO, 
Arleigh Burke Destroyers: Delaying Procurement of DDG 51 Flight III Ships Would Allow 
Time to Increase Design Knowledge, GAO-16-613 (Washington, D.C.: Aug 4, 2016). 

13 Future funding is the amount of investment the portfolio requires to complete all of its 
development activities and to procure all of its planned quantities. 
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• Total development and procurement costs already invested in the 
portfolio have increased from $962 billion to $981 billion over the past 
year. Both invested and future funding needs remain within the 
historical norms. 

• The Navy’s 40 MDAPs comprise 46 percent of the portfolio, but are 
responsible for 60 percent of the portfolio’s total invested development 
and procurement costs. In addition, the Navy’s share of future 
portfolio costs will likely grow because of its plan to grow its fleet size 
by 25 percent, starting in fiscal year 2019 and continuing through 
2048. 14 

• Importantly, our analysis of DOD’s MDAP portfolio excluded classified 
programs, such as the Air Force’s new B-21 Raider program. 


Figure 6 displays the current portfolio’s development and procurement 
funding (invested versus remaining) by military department. 


14 GAO, Navy and Marine Corps: Rebuilding Ship, Submarine, and Aviation Readiness 
Will Require Time and Sustained Management Attention, GAO-19-225T (Washington, 
D.C.: Dec 12, 2018). 
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Figure 6: Major Defense Acquisition Program Investments among Military 
Departments and DOD-wide within the 2018 Portfolio 
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Source: GAO analysis of Department of Defense data. | GAO-19-336SP 

Note: The figure does not account for classified programs, which we excluded from our analyses. The 
figure also excludes $7.17 billion in spent development and procurement funding and $5.04 billion in 
development funding needed to complete the Chemical Demilitarization— Assembled Chemical 
Weapons Alternatives program managed by DOD. 
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A Small Number Of Aging Observation 4. Over the past year, the total estimated acquisition 

Programs Are Driving cost Of the portfolio increased by $26.6 billion, despite its net loss of 

Portfolio Cost Growth four P">srams. 

• One year of cost growth for the current, smaller portfolio (plus the 
addition of two new systems, the Army’s CH-47F Block II and the 
Navy’s T-AO 205) was large enough to offset the cost decrease that 
resulted from the aforementioned exit of six systems, one being the 
Navy’s MH-60R Helicopter, with its estimated cost of $15 billion. 

• Of the $26.6 billion increase, nearly $4 billion is in research and 
development funds, while $21 billion is attributed to procurement 
costs. The remainder ($1.6 billion) is for costs associated with 
acquisition-related operational maintenance and system-specific 
military construction activities. 


Table 1 details the annual change to funding and average cycle times 
between 2017 and 2018 for initial capability delivery portfolios. 


Table 1: Cost and Schedule Changes to DOD’s 2018 Portfolio of 82 Major Defense Acquisition Programs over the Past Year 
(fiscal year 2019 dollars in billions) 


Estimated portfolio 
cost in 2017 

Estimated portfolio 
cost in 2018 

Estimated portfolio 
change since 2017 

Percentage change 
since 2017 

Total estimated research 
and development cost 

300.50 

304.47 

3.97 

1.3 

Total estimated 
procurement cost 

1,344.25 

1,365.29 

21.04 

1.5 

Total estimated acquisition 
cost 

1,659.03 

1,685.65 

26.62 

1.6 

Average cycle time to 
deliver initial capabilities 
(in months) 

120.1 

121.9 

1.8 

1.5 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


Note: In order to make the two portfolios comparable, we added the first full estimates of the two 
entering programs to last year’s portfolio and removed funding and schedule information of the six 
programs that exited the portfolio since last year. 


See appendix V for program- and military department-specific cost 
performance information, including figures related to the performance of 
individual programs over the past year and since first full estimates. 

• Over the past year, 60 programs experienced total acquisition cost 
increases totaling over $41 billion. While most programs realized an 
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increase of less than 5 percent, total cost increases for three 
programs—the Navy’s Ford Class Aircraft Carrier (CVN 78) and DDG 
51 and the Army’s Guided Multiple Launch Rocket Alternative 
Warhead System —comprise 60 percent, or nearly $25 billion, of the 
total growth. In all three cases, the preponderance of cost growth was 
caused by quantity increases. 

• At the same time, 22 programs experienced total acquisition cost 
decreases, totaling $14.5 billion. The Army’s WIN-T Increment 2 
program quantity reductions accounted for $6.3 billion, or 43 percent, 
of that savings. The Army scaled down the program for the same 
reason it canceled WIN-T Increment 3—the platforms’ inability to meet 
the Army’s evolving networking needs. Figure 7 displays the 
portfolio’s total acquisition cost change from 2017 to 2018 by 
percentage change intervals, irrespective of changes in quantity. 


Page 20 


GAO-19-336SP Weapon Systems Annual Assessment 



Figure 7: Total Acquisition Cost Increases and Decreases in Major Defense Acquisition Programs, 2017-2018 
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Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


In December 2008, we reached agreement with DOD and the Office of 
Management and Budget on a set of metrics, which we have used to 
assess DOD’s performance since 2011. These metrics measure cost 
growth of DOD MDAPs since first full estimates and over the last 5-year 
and 1-year periods. Specifically, cost growth of less than 2 percent that 
occurs over a period of 1 year is “acceptable”, while growth occurring 
over 5 years and since program initiation must fall below 10 and 15 
percent, respectively, to meet metric goals. 

DOD officials stated the metrics are dated and do not reflect current 
thinking within the department for tracking cost growth. Consequently, 
they no longer endorse their use. We have yet to agree on an alternative 
way of measuring performance over the specified periods. Such metrics 
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provide DOD with a means to monitor and independently validate the 
effectiveness and sustainability of corrective measures—key criteria 
needed to remove DOD Weapon Systems Acquisition from our high-risk 
list. 15 In July 2018, DOD initiated discussions with us, which will continue 
in 2019, to develop a new set of agreed-upon metrics to track the 
portfolio’s performance over time. We believe the metrics ultimately 
chosen should allow for the department to identify specific programs 
whose costs are approaching unacceptable levels and that may require 
added attention. Until then, we will continue to rely on the previous 
agreed-upon standards, which show that programs continue to fall short 
of the metric for acceptable cost growth over all three recognized periods. 

Figure 8 presents the percentage of programs that satisfied the cost 
growth metrics. As the figure shows, programs have more successfully 
minimized cost growth over 1- and 5-year periods, as compared to cost 
growth since first full estimate. Appendix VI provides additional details on 
the portfolio’s historic cost performance against these metrics. 


15 GAO-19-157SP. 


Page 22 


GAO-19-336SP Weapon Systems Annual Assessment 





Figure 8: Major Defense Acquisition Programs’ Performance against Selected Cost 
Growth Metrics 

Percentage 



1-year 5-year First full estimate 

comparison comparison comparison 
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Source: GAO Analysis of Department of Defense data. | GAO-19-336SP 


MDAP Schedules and 
Costs Have Grown by 35 
Percent and 51 Percent, 
Respectively, since First 
Full Estimates 


Observation 5. Programs in the 2018 portfolio experienced a $569 
billion increase in total acquisition cost since first full estimates and 
average schedule delays of more than 27 months, largely driven by 
aging programs. The majority of cost growth occurred after 
programs began production. 

• Programs’ average delay in delivering initial capabilities has increased 
by over 27 months since their first full estimates. Further, MDAPs 
show, on average, 51 percent total acquisition cost increase since 
their first full estimates, more than half of which is attributable to three 
programs—the Navy’s DDG 51 and SSN 774 and the DOD-wide F-35 
Lightning II Joint Strike Fighter. 

• Programs that have been in the portfolio for over 20 years are the 
primary drivers of these outcomes. Within these programs, average 
delays in initial capability exceed 36 months and cost changes from 
first full estimates have increased by over 83 percent. The portfolio’s 
20 oldest programs have amassed total acquisition cost growth of 
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$474 billion since first full estimates. On the other hand, programs that 
have been in the portfolio for fewer than 10 years recorded 
decreasing costs of $3.4 billion since first full estimates and initial 
capability delays of less than 12 months. 


Table 2 breaks down the portfolio’s total acquisition cost and schedule 
changes since first full estimates, which are not adjusted for quantity 
changes. 


Table 2: Portfolio Cost and Schedule Changes since Major Defense Acquisition Programs’ First Full Estimates (fiscal year 

2019 dollars in billions) 


Dollar change 

Percentage change 

Total research and development cost 

103.49 

51.5 

Total procurement cost 

461.42 

51.0 

Total other (military construction and operations and maintenance) 
acquisition cost 

4.50 

39.6 

Total acquisition cost 

569.41 

51.0 

Average delay in delivering initial capabilities (in months) 

27.4 

34.9 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


While the current portfolio’s costs have grown across all phases of the 
acquisition cycle, the majority of growth occurred after production start. 

As we have previously reported, cost growth during this acquisition phase 
may be symptomatic of programs entering production without attaining 
key knowledge about technology maturity, design stability, and production 
readiness. 16 We assess these parameters later in this report. Sixty-one of 
82 portfolio programs are in production. 

Figure 9 displays the current portfolio’s cost growth across three 
acquisition phases. 


16 GAO, Weapon Systems Annual Assessment: Knowledge Gaps Pose Risks to 
Sustaining Recent Positive Trends , GAO-18-360SP (Washington, D.C.: Apr. 25, 2018). 
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Figure 9: Portfolio Cost Growth by Acquisition Phase (fiscal year 2019 dollars in 
billions) 
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Source: GAO analysis of Department of Defense data. | GAO-19-336SP 

Note: In order to accommodate shipbuilding programs in this analysis, we correlated detail design 
contract awards, fabrication starts, and lead ship deliveries with development start, critical design 
review, and production start, respectively. 


Quantity Increases and Cost Reporting 

In the cases of SBIRS High and AEHF, the Air 
Force increased quantities by two additional 
units late into each programs’ acquisition 
cycle, which the Air Force called a “block 
buy.” Each program reported separate cost 
and funding information for the block buys 
and eventually removed the first four satellite 
units in each program’s Selected Acquisition 
Report after they were 90 percent complete. 
This reduced our visibility into each program’s 
cost performance and, as a result, our 
numbers do not reflect DOD’s total investment 
in them. 


Other contributing factors to high cost growth after programs enter 
production include increases in quantities or added capabilities after first 
full estimates are finalized. For example, satellite systems such as the Air 
Force’s Space-Based Infrared System (SBIRS) High and the Advanced 
Extremely High Frequency (AEHF) added additional satellite units in the 
form of “block buy” subprograms well after the programs’ first full 
estimates. 


Source: GAO analysis of Department of Defense data. | GAO 
19-336SP 


Programs’ Cost 
Performance Varied 
Considerably When 
Controlling for Quantity 
Changes 


Observation 6. For some programs, cost performance reversed 
when factoring in quantity changes. 

We analyzed the total amount of acquisition cost changes, not due to 
quantity changes, since first full estimates. We then ranked the programs 
in terms of highest and lowest performing, by both total dollars and total 
percentage, in tables 3 and 4 below. 
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• Navy programs showed the widest variability in cost changes, with the 
best performing program and the second-worst performing program 
when adjusted for quantities. 

• Some programs showed improved cost performance when we 
accounted for quantity changes, as was the case of the Navy’s DDG 
51 program. This program was one of three that drove overall portfolio 
cost growth since first full estimate. Yet, when we accounted for 
quantity changes, our analysis showed program cost reductions of 
more than $25 billion since first full estimate. Likewise, the Air Force 
Joint Direct Attack Munition (JDAM) program’s unadjusted total cost 
growth since first full estimate is almost $8.8 billion. When adjusted for 
quantities, it had a cost reduction of more than $3.3 billion since its 
first full estimate. 

• At least one program, the Navy’s DDG 1000, showed decreased 
costs, but not at a rate commensurate with the decreased quantities. 

In 1997, the Navy set an initial baseline for the program that required 
a quantity of 32 ships. In 2008, the Navy truncated the program at 
three ships. This substantial reduction in quantity has driven an 
overall reduction in program costs totaling $13.5 billion. Nonetheless, 
the Navy is now only procuring three ships for what it originally 
planned to spend on 20 ships. 


Tables 3 and 4 rank the 82 current MDAPs by the 10 programs with the 
lowest and highest total acquisition cost changes—unrelated to quantity 
changes—since programs’ first full estimates. Table 3 ranks the programs 
by changes in total dollars, while Table 4 ranks the programs by total 
percentage changes. 
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Table 3: Ten Major Defense Acquisition Programs with the Lowest and Highest Total Acquisition Cost Change since First Full 
Estimates (fiscal year 2019 dollars in millions) 

10 Programs with the Lowest Total Acquisition Cost Changes Since First Full Estimate 

Rank 

Program name 

Cost change 

Lead 

component 

Program start 
date 

1 . 

DDG51 

-$25,066.81 

Navy 

5/15/1983 

2. 

KC-46A 

-$7,851.91 

Air Force 

2/15/2011 

3. 

Multifunctional Information Distribution System 

-$5,039.60 

Navy 

12/15/1993 

4. 

Joint Direct Attack Munition 

-$3,346.45 

Air Force 

10/15/1993 

5. 

Joint Light Tactical Vehicle 

-$2,761.52 

Army 

12/22/2007 

6. 

Airborne & Maritime/Fixed Station 

-$2,407.79 

Army 

12/15/2007 

7. 

CVN 78 

-$2,213.62 

Navy 

4/23/2004 

8. 

HC/MC-130 Recap 

-$1,577.44 

Air Force 

2/15/2010 

9. 

AH-64E New Build 

-$930.40 

Army 

7/15/2010 

10. 

AIM-9X Blkll 

-$871.69 

Navy 

6/15/2011 

10 Programs with the Highest Total Acquisition Cost Changes Since First Full Estimate 

Rank 

Program name 

Cost change 

Lead 

component 

Program start 
date 

1 . 

F-35 

$147,689.74 

Joint Department 
of Defense- 
(DOD) wide 

11/15/1996 

2. 

V-22 

$40,945.63 

Navy 

12/15/1982 

3. 

Evolved Expendable Launch Vehicle 

$39,607.54 

Air Force 

12/15/1996 

4. 

SSN 774 

$34,542.19 

Navy 

8/15/1994 

5. 

DDG 1000 

$19,451.31 

Navy 

12/15/1997 

6. 

Trident II 

$16,068.45 

Navy 

10/15/1977 

7. 

Advanced Medium Range Air-to-Air Missile 

$13,427.50 

Air Force 

11/15/1978 

8. 

UH-60M Black Hawk 

$11,786.87 

Army 

4/15/2001 

9. 

Guided Multiple Launch Rocket System and Alternate 
Warhead 

$11,234.73 

Army 

3/15/1998 

10. 

Chemical Demilitarization—Assembled Chemical Weapons 
Alternatives 

$10,783.10 

Joint DOD-wide 

3/31/1998 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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Table 4: Ten Major Defense Acquisition Programs with the Lowest and Highest Total Acquisition Cost Percentage Change 
since First Full Estimates 


10 Programs with the Lowest Total Acquisition Cost Percentage Changes since First Full Estimate 

Rank 

Program name 

Percentage 

change 

Lead 

component 

Program 
start date 

1 

Multifunctional Information Distribution System 

-345 

Navy 

12/15/1993 

2 

DDG51 

-147 

Navy 

5/15/1983 

3 

Joint Direct Attack Munition 

-87 

Air Force 

10/15/1993 

4 

AH-64E New Build 

-35 

Army 

7/15/2010 

5 

Airborne & Maritime/Fixed Station 

-26 

Army 

12/15/2007 

6 

AIM-9X Blkll 

-19 

Navy 

6/15/2011 

7 

HC/MC-130 Recap 

-17 

Air Force 

2/15/2010 

8 

KC-46A 

-16 

Air Force 

2/15/2011 

9 

B61 Mod 12 LEPTKA 

-15 

Air Force 

11/15/2012 

10 

Small Diameter Bomb II 

-14 

Air Force 

7/29/2010 

10 Programs with the Highest Total Acquisition Cost Percentage Changes since First Full 

Rank 

Program name 

Percentage 

change 

Lead 

component 

Program 
start date 

1 

Guided Multiple Launch Rocket System and Alternate 
Warhead 

567 

Army 

3/15/1998 

2 

Chemical Demilitarization—Assembled Chemical 
Weapons Alternatives 

364 

Joint 

Department of 

Defense- 
(DOD) wide 

3/31/1998 

3 

CH-47F 

280 

Army 

11/15/1997 

4 

MQ-1C Gray Eagle 

263 

Army 

4/15/2005 

5 

H-1 Upgrades 

216 

Navy 

9/15/1996 

T6 

Littoral Combat Ship 

204 

Navy 

5/15/2004 

T6 

Evolved Expendable Launch Vehicle 

204 

Air Force 

12/15/1996 

8 

Ground/Air Task Oriented Radar 

111 

Navy 

3/15/2005 

9 

Advanced Medium Range Air-to-Air Missile 

110 

Air Force 

11/15/1978 

10 

MQ-9 Reaper 

108 

Air Force 

2/15/2004 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


Appendix VII contains an expanded set of rankings that utilize the same 
criteria as the tables above, but identifies the best/worst performing five 
programs within each individual military department. 
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Programs Initiated Since 
2010 Continue to 
Outperform Others, but 
Positive Performance 
Continues to Slow 


Observation 7. Programs initiated since 2010 have realized a total 
acquisition cost increase of $1.4 billion since last year, whereas last 
year we found no growth in these programs. Those initiated before 
2010 recorded an increase of $25.2 billion over the same period. 

Table 5 outlines changes in both the total estimated acquisition cost and 
the average cycle time to deliver initial capabilities for the programs 
initiated before and since 2010. 
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Table 5: Programs’ Total Acquisition Cost and Cycle Time Changes, 2017 to 2018 (fiscal year 2019 dollars in billions) 



Estimated portfolio 
cost in 2017 

Estimated portfolio 
cost in 2018 

Estimated portfolio 
change since 2017 

Percentage change 
since 2017 

Programs initiated since 2010 

Total estimated research and 
development cost 

55.71 

57.07 


1.36 

2.4 

Total estimated procurement cost 

219.18 

219.06 


-0.11 

-0.1 

Total estimated acquisition cost 

278.41 

279.84 


1.43 

0.5 

Average cycle time to deliver initial 
capabilities (in months) 

92.2 

93.7 


1.5 

1.6 

Programs initiated before 2010 

Total estimated research and 
development cost 

244.79 

247.40 


2.61 

1.1 

Total estimated procurement cost 

1,125.07 

1,146.22 


21.15 

1.9 

Total estimated acquisition cost 

1,380.61 

1,405.80 


25.19 

1.8 

Average cycle time to deliver initial 
capabilities (in months) 

134.8 

136.5 


1.7 

1.3 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


While the two groups of programs initiated before and since 2010 both 
experienced total acquisition cost increases since last year, the 
percentage increase for older programs is more than triple that of the 
newer programs. This performance disparity supports the observation 
from prior years that acquisition reforms, namely the Weapon Systems 
Acquisition Reform Act of 2009 and three Better Buying Power initiatives, 
have driven reduced cost growth. 17 On the other hand, the cost growth 
now realized by the newer programs reflects a reversal since our prior 
assessment, when this group showed overall cost decreases. Over the 
past year, we observed a total of $1.4 billion in cost growth among these 
newer programs. Thus, instead of helping offset cost increases within the 
portfolio, as they have done in the past, newer programs directly 
contributed to the portfolio’s cost growth between 2017 and 2018. 


17 Weapon Systems Acquisition Reform Act of 2009, Pub. L. No. 111-23; Between 2010 
and 2015, the Under Secretary of Defense for Acquisition, Technology, and Logistics 
implemented “Better Buying Power” initiatives based on the principle that continuous 
improvement is the best approach to improving the performance of the defense acquisition 
enterprise. 
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Similarly, while newer programs continue to minimize short-term cost 
growth more efficiently than older programs, that effectiveness has 
diminished. In particular, newer programs as a whole show additional cost 
growth between development start and critical design review (CDR) than 
in our prior assessment. This is likely because half of the newer programs 
have already passed, or are approaching, CDR. Figure 10 shows the cost 
performance by phase for both newer and older programs, as measured 
since first full estimate. 


Figure 10: Programs’ Total Acquisition Cost Changes since First Full Estimate by 
Acquisition Phases 


Fiscal year 2019 dollars (in billions) 


Program initiated since 2010 (28 programs) 


Program initiated prior to 2010 (54 programs) 


Development start to 
critical design review 


$.485 


$77.48 


Critical design review 
to production decision 


-$3.54 


$116.86 


Production start to 
end of production 


-$3.00 


$281.15 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


Note: To accommodate shipbuilding programs in this analysis, we correlated detail design contract 
awards, fabrication starts, and lead ship deliveries with development start, critical design review, and 
production start, respectively. 


Our analysis further showed that: 

• Overall, the current cost estimates for the 28 programs initiated since 
2010 have decreased from their first full estimates, which is illustrated 
above as cost decreases incurred after both CDR and production 
start. While we acknowledge the decreases in projected future 
spending, only 15 of the 28 since-2010 programs have conducted 
their CDR, and only 11 are in production, which is the phase where 
cost growth has historically been the greatest. At this point, it is 
unclear whether the newer programs can maintain lower spending 
projections as they proceed through the acquisition process. 

• Despite the positive changes to cost projections beyond CDR and 
production for since-2010 programs, those same programs reported 
cost growth of $485 million between development start and CDR, 
almost doubling the growth we identified last year over the same 
acquisition timeframe. 
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In comparison, cost growth reported by the 54 programs initiated prior 
to 2010 occurred during all phases of the acquisition process, with the 
vast majority occurring after CDR or start of production. 


Figure 11 illustrates how the current portfolio’s total acquisition cost is 
apportioned between older programs that were initiated prior to 2010, 
newer ones initiated since 2010, and the F-35 Lightning II Joint Strike 
Fighter—DOD’s largest acquisition program, initiated in 2001. 


Figure 11: Distribution of Total Acquisition Costs among Various Groups of 
Programs 

Fiscal year 2019 dollars (in millions) 



$279.84 

Programs initiated since 2010 (28) 


$360.10 

F-35 (initiated before 2010) 


$1,045.70 

Programs initiated before 2010 (53) 


Source: GAO Analysis of Department of Defense data. | GAO-19-336SP 


• The deteriorating performance of newer programs can also be seen in 
their diminished buying power gains. Buying power is the amount of 
goods or services that can be purchased given a specified level of 
funding. To determine changes in buying power, the effects of 
quantity changes must be isolated from other factors that affect cost. 

• A program’s cost can increase because of additional quantities. While 
that does represent a cost increase, it does not necessarily indicate 
acquisition problems or a loss of buying power. Alternatively, a 
program’s cost can decrease due to a reduction in quantity and it may 
still experience a buying power gain or loss. 
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• While newer, post-2010 programs still recorded a buying power gain 
of $263 million since last year, it is merely a fraction of past gains. For 
instance, post-2010 programs’ buying power gains amounted to 
nearly $5 billion in 2017. 

• On the other hand, the portfolio’s overall buying power gain of $3.9 
billion was largely driven by the performance of older programs, 16 of 
which experienced a combined buying power increase of nearly $11 
billion. 

Table 6 showcases buying power changes across the two groups of 
programs. Negative numbers indicate decreased costs and a gain in 
buying power, while positive numbers indicate the opposite. 
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Table 6: Buying Power Changes Since 2017 (fiscal year 2019 dollars in millions) 

Buying Power since 2017 for Programs Initiated before 2010 


Number of 
programs 

Actual 
procurement cost 
change 

GAO calculated 
cost change 
attributable to 
quantity changes 

GAO calculated 
cost change not 
attributable to 
quantity changes 

Increased buying power 


16 

9,525.75 

20,508.76 

-10,983.01 

Procurement cost decreased with no quantity 
change 


7 

-4,178.24 

0.00 

-4,178.24 

Quantity increased with less cost increase than 
anticipated 


8 

20,299.79 

26,812.46 

-6,512.67 

Quantity decreased with more cost decrease than 
anticipated 


1 

-6,516.96 

-6,303.70 

-213.26 

Decreased buying power 


36 

11,621.06 

4,352.69 

7,268.37 

Procurement cost increased with no quantity change 


23 

3,213.37 

0.00 

3,213.37 

Quantity increased with more cost increase than 
anticipated 


6 

11,534.59 

9,511.20 

2,023.39 

Quantity decreased with less cost decrease than 
anticipated 


7 

-3,205.74 

-5,158.51 

1,952.77 


No change in buying power 


2 

0.00 

0.00 

0.00 

Sub-portfolio Totals 


54 

21,146.81 

24,861.45 

-3,714.64 

Buying Power since 2017 for Programs Initiated since 2010 

Increased buying power 


9 

-1,178.12 

76.92 

-1,255.04 

Procurement cost decreased with no quantity 
change 


6 

-1,064.50 

0.00 

-$1,064.50 

Quantity increased with less cost increase than 
anticipated 


2 

55.46 

173.81 

$118.35 

Quantity decreased with more cost decrease than 
anticipated 


1 

-169.08 

-96.89 

$72.19 

Decreased buying power 


17 

1,066.87 

75.02 

991.85 

Procurement cost increased with no quantity change 


16 

952.58 

0.00 

952.58 

Quantity increased with more cost increase than 
anticipated 


1 

114.29 

75.02 

39.27 

Quantity decreased with less cost decrease than 
anticipated 


0 

0.00 

0.00 

0.00 

No change in buying power 


2 

0.00 

0.00 

0.00 

Sub-portfolio Totals 


28 

-111.25 

151.94 

-263.19 

Portfolio Totals 


82 

21,035.56 

25,013.39 

-3,977.83 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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DOD MDAPs Operate in 
an Environment of Limited 
Competition and a 
Constrained Defense 
Contractor Base 


Observation 8. Current MDAPs did not compete 67 percent of their 
currently reported major development and procurement contracts. 

We have reported that competition is the cornerstone of a sound 
acquisition process and a critical tool for achieving the best return on 
investment for taxpayers. 18 Generally, a low competition rate can 
contribute to increased costs of goods and services and decreased 
buying power. According to the Office of Federal Procurement Policy, 
“ineffective competition”—when one offer is received in response to a 
competitive solicitation—deprives the programs and their managing 
agencies of the ability to consider alternative solutions in a reasoned and 
structured manner, thereby creating opportunities for increased costs, 
substandard products and prolonged acquisition cycles. In July 2009, the 
Office of Management and Budget (OMB) issued a memorandum to 
department and agency heads in which it categorized noncompetitive 
contracts as high-risk because, absent competition, agencies must 
negotiate contracts without the benefit of a direct market mechanism to 
help set the contract price. 


Our prior work has identified various factors that affect competition rates, 
including the government’s preference for a specific vendor, inadequate 
acquisition planning, and overly restrictive government requirements. 
Moreover, we identified DOD-specific factors, such as reliance on original 
manufacturers and entry barriers resulting from the complexity of DOD’s 
contracting process and lack of data rights—legal rights a contractor 
and/or the government have to use technical, intellectual property and 
software data—as contributing to DOD’s particularly low competition 
rates. 19 


Creation and sustainment of competitive environments was a recurring 
theme in DOD’s Better Buying Power policy memorandums issued 
between 2010 and 2015. In order to analyze DOD’s performance in 
generating competition in contracting within the 2018 MDAP portfolio, we 
compiled a list of all 183 active major contracts and orders identified in 
individual programs’ 2017 SARs and compared them to information in the 


18 GAO, Defense Contracting: Early Attention in the Acquisition Process Needed to 
Enhance Competition, GAO-14-395 (Washington, D.C.: May 5, 2014). 

19 GAO, Federal Acquisitions: Congress and the Executive Branch Have Taken Steps to 
Address Key Issues, but Challenges Endure, GAO-18-627 (Washington, D.C.: Sept. 12, 
2018). 
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FPDS-NG database. 20 All but three of these contracts and orders were for 
development or procurement. 21 Of the 183 contracts and orders included 
in our review: 

• the department awarded 180 for development and procurement 
efforts, 

• the Navy awarded 85 contracts that totaled $147.1 billion, 

• the Air Force awarded 51 contracts that totaled $132.6 billion, 

• the Army awarded 37 contracts that totaled $50.5 billion, and 

• Joint, DOD-wide offices awarded 10 contracts that totaled $33.8 
billion. 

Figure 12 shows the percentage of the 183 contracts in our analysis that 
DOD either competed or did not compete, as well as the reported number 
of offers DOD received for each category. Appendix II provides more 
details on the selection of contract actions included in this analysis, as 
well as how we define “competitive” and “non-competitive” actions. 


20 By statute, SARs include an MDAP’s “major contracts,” which are defined as “each of 
the six largest prime, associate, or Government-furnished equipment contracts under the 
program that is in excess of $40,000,000 and that is not a firm, fixed price contract.” An 
MDAP’s SAR requirements cease after 90 percent of the total quantity of items to be 
purchased under the program are delivered or 90 percent of planned expenditures are 
made. DOD guidance states that once a contract is 90 percent complete, it is no longer 
reported in the SAR. 

21 Of the 183 contracts and orders in our analysis, DOD reporting indicated that 115 were 
for procurement; 65 were for research, development, test, and evaluation; two were for 
acquisition-related operations and maintenance; and one was for system-specific military 
construction. 
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Figure 12: Competition in Currently Reported Major Contracts and Number of Offers 
Received 



Source: GAO analysis of Federal Procurement Data System—Next Generation data. | GAO-19-336SP 

Note: In order to analyze DOD’s performance in generating competition in contracting within the 2018 
major defense acquisition program portfolio, we compiled all 183 contracts listed on individual 
programs’ 2017 SARs and compared them to information in the Federal Procurement Data System- 
Next Generation database. 


Federal statutes and acquisition regulations generally require that 
contracts be awarded on the basis of full and open competition. However, 
they also permit agencies to award noncompetitive contracts in certain 
circumstances, such as: 

• only one responsible source and no other supplies or services will 
satisfy agency requirements; 

• unusual and compelling urgency; and 

• when authorized or required by statute (for example, sole source 
awards to small businesses.) 22 

Additionally, Federal Acquisition Regulation (FAR) part 34, which 
establishes policies for the acquisition of major systems, directs agencies 
and program managers to promote and sustain effective competition 
among alternative system concepts and sources for “as long as it is 


22 FAR §§ 6.302-1,6.302-2, 6.302-5. 
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economically beneficial and practicable to do so." 23 Thus, the FAR 
reflects that there may be a point in a major system acquisition at which 
promoting and sustaining competition among alternative sources is no 
longer economically beneficial and practicable. 

Our analysis of the 183 contract awards also found the following: 

• Competition rates for research and development contract awards 
were considerably higher, at 81 percent competed, than those for 
procurement contract awards, where the competition rate was barely 
over 15 percent. This dichotomy may be linked to the government’s 
tendency to rely on original weapon systems equipment 
manufacturers for follow-on procurements, to include sustainment 
contracts. 24 

• In March 2017, we reported on overall trends in defense and civilian 
agencies’ contract obligations, such as trends in competition. 25 In that 
report, we noted that DOD produces annual competition reports, 
which recognize that competition achievement varies by organization 
based on the mission and types of products and services being 
procured. For example, competition rates are lower in organizations 
that buy major weapon systems—such as those covered in this 
annual assessment—particularly as systems move into production. 


23 FAR §§ 34.002(b), 34.005-1 (a). In FAR part 34, “effective competition” is defined as “a 
market condition that exists when two or more contractors, acting independently, actively 
contend for the Government's business in a manner that ensures that the Government will 
be offered the lowest cost or price alternative or best technical design meeting its 
minimum needs.” FAR § 34.001. 

24 The additional systems or sustainment are often procured through contract modifications 
or the exercise of contract options. This situation is partly attributable to the unique 
relationship that DOD has with the defense industry that differs from the commercial 
marketplace. The combination of a single buyer (DOD), few very large prime contractors 
in each segment of the industry, and a limited number of weapon programs constitutes a 
structure for doing business that is altogether different from a classic free market. For 
instance, there is less competition and once a contract is awarded, the contractor often 
remains the sole vendor capable of providing additional systems and sustainment. These 
long-term contractual relationships with weapon system contractors limit opportunities for 
competition. See GAO-14-395. 

25 GAO, Contracting Data Analysis: Assessment of Government-wide Trends, 

GAO-17-244SP (Washington, D.C.: Mar. 9, 2017). 
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Figure 13 displays the percentage of the 183 contracts that DOD awarded 
competitively and noncompetitively—by military department and Joint, 
DOD-wide—we analyzed. 


Figure 13: Competition Rates for 183 Currently Reported Major Contract Awards 
within DOD’s 2018 Portfolio of Major Defense Acquisition Programs 

Percentage 



Air Force Army Joint DOD-wide Navy 

Department 



Not competed 
Competed 


Source: GAO analysis of Federal Procurement Data System—Next Generation data. | GAO-19-336SP 


Figure 14 shows total dollar amounts of 180 major development and 
procurement contracts that DOD awarded competitively and 
noncompetitively—by military department and Joint, DOD-wide. 
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Figure 14: Dollars Associated with Competition Results for 180 Currently Reported Major Development and Procurement 
Contract Awards within DOD’s 2018 Portfolio of Major Defense Acquisition Programs 


Fiscal year 2019 dollars (in billions) 
100 


80 



$23 



Development Procurement 
Joint DOD-wide 


$93 




Competed 
Not competed 


Source: GAO analysis of Federal Procurement Data System—Next Generation data. | GAO-19-336SP 


A Small Group of 
Contractors Received the 
Majority of MDAP Contract 
Award Dollars 


Observation 9. DOD awarded 47 percent of its currently reported 183 
major development and procurement contract awards to five 
corporations and entities connected with them. This 47 percent 
constituted 72 percent of the dollars associated with the 183 major 
contracts; the remaining 28 percent was distributed among 30 other 
companies. 


• Of the 183 major contract awards we analyzed, five corporations and 
entities connected with them received a combined 86 awards totaling 
over $262 billion. The prime contractors are Boeing, Lockheed Martin, 
General Dynamics, United Technologies, and Northrop Grumman. 
DOD awarded 97 contracts totaling almost $100 billion to 30 other 
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companies. 


Figure 15 shows how DOD divided the 183 contract awards we reviewed 
among the top five prime contractors and remaining companies. Figure 
16 shows the total dollars associated with these contract awards by prime 
contractor. 


Figure 15: Distribution of Contract Awards among Prime Contractors for 183 
Currently Reported Major Contracts within DOD’s 2018 Portfolio of Major Defense 
Acquisition Programs 



5 contract awards 

United Technologies Corporation 

11 contract awards 

Northrop Grumman Corporation 

11 contract awards 

General Dynamics Corporation 

23 contract awards 

The Boeing Company 


36 contract awards 

Lockheed Martin Corporation 


97 contract awards 

Remaining companies (30) 


Source: GAO analysis of Federal Procurement Data System—Next Generation data. | GAO-19-336SP 


Note: Percentages for each of the five prime contractors identified in the figure also include awards to 
entities connected with them. Values may not sum to 100 percent due to rounding. 
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Figure 16: Distribution of Dollars among Prime Contractors for 183 Currently 
Reported Major Contracts within DOD’s 2018 Portfolio of Major Defense Acquisition 
Programs 
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contractors 
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The Boeing Company 
Lockheed Martin Corporation 
General Dynamics Coporation 
United Technologies Corporation 
Northrop Grumman Corporation 


Source: GAO analysis of Federal Procurement Data System—Next Generation data. | GAO-19-336SP 

Notes: Dollar values for each of the five prime contractors identified in the figure also include those 
from entities connected with them. These contract values include the base contract and all options, 
regardless of whether the option has been, or will be, exercised. 


Our Observation on 
DOD’s Application of 
Key Acquisition 
Reforms 


Our analysis of questionnaire responses from 51 MDAPs shows that 
these programs have applied, to varying degrees, six selected acquisition 
reforms included in NDAAs between fiscal years 2016 and 2018. We 
completed this analysis to identify the extent to which reforms have taken 
hold across DOD, not to evaluate programs’ compliance in implementing 
them. Prior to passage of these newer acquisition reforms, numerous 
programs had already taken steps related to the areas addressed by 
some of the reforms. For instance, many programs that were well into 
either development or production had crafted an acquisition strategy that 
called for the use of a modular, open-systems design prior to the related 
reform described below. Others had previously established goals for 
numerous acquisition elements, such as procurement unit costs, 
sustainment costs, and an initial operational capability date. However, 
these reforms are still relatively new, and hence will apply primarily to 
newer systems that have yet to begin development. In other cases, the 
reforms offer options—as opposed to requirements—for the department 
and programs to consider utilizing during program execution. 
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Observation 1. Six acquisition reforms we reviewed have gained 
varying degrees of traction within the 51 programs we surveyed; 
even though many of these programs predate these reforms, 
programs have taken action in areas related to the reforms. 

Forty-five of the 51 programs are well into either development or 
production and beyond several of the key acquisition phases where 
application of many of the reforms might be expected to occur. At the 
same time, most, but not all, of the six reforms we reviewed built upon 
concepts that already existed in policy or law. For example, DOD’s Better 
Buying Power initiatives have emphasized the use of modular, open 
systems architecture. 

Table 7 presents an overview of the six acquisition reforms we included in 
our analysis. Appendix IV contains more detailed descriptions of these 
acquisition reforms. 


Table 7: Overview of Selected, Recent Acquisition Reforms 

Legislative 

provision 

Name of reform 

Description 

Fiscal Year (FY) 2016 
National Defense 
Authorization Act 
(NDAA) § 804 

Middle tier of acquisition for rapid 
prototyping and rapid fielding 

Calls for the Department of Defense (DOD) to issue guidance for 
a “middle tier” of acquisition programs to be completed in two to 
five years through two types of “acquisition pathways:” rapid 
prototyping and rapid fielding. 

FY 2016 NDAA §806 

Secretary of Defense waiver of acquisition 
laws to acquire vital national security 
capabilities 

Authorizes the Secretary of Defense to waive any provision of 
certain types of acquisition laws and regulations to, among other 
things, meet vital national security needs. 

FY 2016 NDAA § 

823 a 

Revision of Milestone A decision authority 
responsibilities for major defense 
acquisition programs (MDAP) 

Requires a milestone decision authority to make various written 
determinations before granting Milestone A approval, including 
that an MDAP: 

(1) fulfills an approved Initial Capabilities Document, 

(2) is being developed in light of appropriate market research, 

(3) has a risk reduction plan for any identified areas of risk 

(4) addresses sustainment planning, and 

(5) has a cost estimate and the level of resources is sufficient for 
successful program execution. 

FY 2017 NDAA §805 

Modular, open-systems approach in 
development of MDAPs 

Requires that MDAPs that receive Milestone A or Milestone B 
approval after January 1,2019 must be designed and developed, 
to the maximum extent practicable, with a modular, open- 
systems approach to enable incremental development and 
enhance competition, innovation, and interoperability. 
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Legislative 

provision 

Name of reform 

Description 


Requires that (1) DOD set goals for MDAPs’: 

a) procurement unit cost, 

b) sustainment costs, 

c) initial operational capability date, 

d) technology maturity, and 

e) prototyping, 

and (2) ensure that MDAPs complete an independent technical 
risk assessment 0 

FY 2018 NDAA § 864 Other transaction (OT) authority for certain Increased the service acquisition executive and Under Secretary 
prototype projects of Defense approval thresholds for use of OTs for prototype 

projects to $100 million and $500 million, respectively. 

Source: GAO analysis of the NDAAs for fiscal years 2016-2018 and of MDAP responses to GAO’s questionnaire. | GAO-19-336SP 

a Prior legislation required a milestone decision authority to certify to some of the same or similar 
elements as those in FY2015 NDAA § 823. See 10 U.S.C. 2366a (2014), amended by FY17 NDAA, 
Pub. L. No. 114-92, § 823 (2015). 

b We did not include Air Force systems within the tallies for this acquisition reform. 

c We collected data associated with this acquisition reform, but later identified information that led us 
to conclude this data is unreliable. As a result, we excluded this information from our analysis and we 
will obtain this information for future reporting. 


FY 2017 NDAA § 807 Cost, schedule and performance goals of 
MDAPs b 


Table 8 illustrates instances where programs reported taking actions in an 
area addressed by the acquisition reforms. In certain cases, these actions 
preceded the reforms. 


Page 44 


GAO-19-336SP Weapon Systems Annual Assessment 




Table 8: Actions in Areas Related to Recent Acquisition Reforms as Reported by 51 Current and Future Major Defense 
Acquisition Programs (MDAP) 

Areas addressed by acquisition reform Number of 

MDAPs that 
reported actions 
in the area 

Secretary of Defense waivers of acquisition laws and regulations 3 

Revision of milestone A decision authority responsibilities for MDAPs, to include milestone decision authority 

written determinations that the program g 

1. fulfilled an approved Initial Capabilities Document, 5 

2. is being developed in light of appropriate market research, g 

3. has a risk reduction plan in place for any identified areas of risk g 

4. addresses sustainment planning, and 7 

5. has a cost estimate and the level of resources is sufficient for successful program execution 3 


Modular, open-systems approach in development of MDAPs 33 

Cost, schedule, and performance goals of MDAPs’: 

a. procurement unit cost, 22 

b. sustainment costs, 21 

c. initial operational capability date, 29 

d. technology maturity, and 12 

e. prototyping, 7 

and ensure that MDAPs complete an independent technical risk assessment — 

Other transactions authority for certain prototype projects 2 

Middle tier of acquisition for rapid prototyping and rapid fielding 13 0 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


a Prior legislation required a milestone decision authority to certify to some of the same or similar 
elements as those in FY2015 NDAA § 823. See 10 U.S.C. 2366a (2014), amended by FY17 NDAA, 
Pub. L. No. 114-92, § 823 (2015 ). While many programs provided questionnaire responses regarding 
areas addressed by the acquisition reforms, only 22 current MDAPs reported having held a milestone 
A review. Further, only one program (Improved Turbine Engine Program) held its milestone A review 
after enactment of the FY16 NDAA. One pre-MDAP (Precision Strike Missile) was initiated at 
milestone A, and this also occurred after enactment of the FY16 NDAA. 

b The statute that defines what constitutes an MDAP states that MDAPs do not include programs or 
projects carried out using the middle tier rapid prototyping or rapid fielding acquisition pathway. 10 
U.S.C. § 2430(a)(2)(A). However, section 804 of the FY16 NDAA contemplates that the rapid 
prototyping pathway will include a process for transitioning successful prototypes to new or existing 
programs for production and fielding, under either the rapid fielding pathway or the traditional 
acquisition system. FY16 NDAA, Pub. L. No. 114-92, § 804 (2015) (codified, as amended, at 10 
U.S.C. § 2302 note). 

Some examples of programs’ reports regarding areas addressed by these 
acquisition reforms include the following: 

• The Army’s Improved Turbine Engine Program was the only program 
among 45 current MDAPs we surveyed that was scheduled to enter 
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system development after January 1, 2019. The program reported 
having an acquisition strategy with a modular, open-systems 
approach, as well as applying many features in the acquisition reform 
provisions cited in our questionnaire. 

• None of the 45 current MDAPs in our review reported using any other 
transaction authority instruments in connection with Section 864 of the 
NDAA for Fiscal Year 2018. Only two of the 6 future MDAPs reported 
using other transaction agreements. The Army’s Precision Strike 
Missile reported awarding two separate OTs totaling $240.5 million, 
although the awards occurred prior to enactment of this provision. 

• Although our other ongoing work has identified that DOD has begun 
over 30 middle-tier acquisition programs under the authority granted 
in DOD guidance implementing Section 804 of the NDAA for Fiscal 
Year 2016, MDAPs did not report any plans to use prototypes 
associated with this authority. For all 51 current and future MDAPs we 
surveyed, program officials could not identify any middle tier 
prototypes that, as of February, 2019, had transitioned into their 
offices for production or fielding. Further, for the 45 current MDAPs we 
surveyed, officials from 44 of the programs reported that they either 
did not anticipate receiving any such prototypes in time for critical 
design review or production decision, or that the question was not 
applicable. Only one program reported that it did not know whether it 
would receive such prototypes in the future. 

We have ongoing work to assess DOD’s efforts to more broadly 
implement many of these acquisition reforms. We will continue to assess 
DOD’s progress going forward. 


Eight Observations 
from Our Assessment 
of Knowledge 
Attained at Key 
Decision Points and 
Related Implications 
for Testing and 
Software 
Development 


Based on our analysis of 51 selected weapon acquisition programs, we 
found that while a few programs implemented elements of knowledge- 
based acquisition practices, implementation across the entire portfolio 
remains inconsistent. Numerous acquisition programs, both current and 
future, have proceeded, or plan to proceed, through key acquisition points 
without the requisite knowledge called for in best practices. For instance, 
several programs proceeded through key milestones without attaining the 
knowledge recommended for the event—such as maturing technologies 
and conducting a preliminary design review prior to starting system 
development, or testing a system-level integrated prototype to inform a 
critical design review. Several programs do not plan to deliver a baseline 
software capability until after production start, and many others plan to 
declare an initial operational capability in advance of testing. Actions like 
these increase the risk of cost growth and schedule delays. 


Page 46 


GAO-19-336SP Weapon Systems Annual Assessment 



For the second year in a row, we analyzed programs that have 
progressed through all three knowledge points—development start, 
critical design review, and a production decision. This year’s analysis 
includes 17 programs for which we have previously collected information 
for prior assessments. This year our analysis revealed that MDAPs that 
implemented key knowledge practices saw substantially better cost 
performance and less schedule growth than programs that have not 
implemented those same practices. This analysis emphasizes the 
importance of knowledge-based acquisition practices and the positive 
effects they can have on acquisition outcomes. 


Current Programs Are 
Inconsistently Following 
Knowledge-Based 
Approaches 


1. MDAPs have inconsistently applied key knowledge practices, 
even newer programs that have more recently progressed through 
early acquisition events. Few programs held preliminary design 
reviews in advance of development start decisions or demonstrated 
critical technologies in a realistic environment. 

Table 9 presents key knowledge practices and the percent of current 
acquisition programs that attained the requisite knowledge for each 
practice. 


Table 9: Major Defense Programs Have Not Consistently Implemented Knowledge-Based Acquisition Practices 

Practices associated with the three 

Programs that completed the KP before 

Programs that completed the KP during 

key knowledge points (KP) 

this assessment period 

this assessment period 

KP 1 practices 

38 programs 

Four programs 

Demonstrate all critical technologies are 
very close to final form, fit, and function 
within a relevant environment 

€ 

€ 

Demonstrate all critical technologies are 
in form, fit, and, function within a 
realistic environment 

O 

O 

Completed preliminary design review 
before system development start 

€ 

O 

KP 2 practices 

33 programs 

Two programs 

Release at least 90 percent of design 
drawings to manufacturing 

o 

• 

Test a system-level integrated 
prototype 

o 

o 

KP 3 practices 

15 programs 

Three programs 

Demonstrate critical manufacturing 
processes are in statistical control 

o 

o 

Demonstrate critical processes on a 
pilot production line 

€ 

• 
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Practices associated with the three 
key knowledge points (KP) 

Programs that completed the KP before 
this assessment period 

Programs that completed the KP during 
this assessment period 

Test a production-representative 
prototype in its intended environment 

o 

• 


Legend: 

• Practice implemented in 75- 100 percent of programs 
© Practice implemented in 50 - 74 percent of programs 
o Practice implemented in 0 - 49 percent of programs 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 

Notes: To accommodate shipbuilding programs in this analysis, we correlated detail design contract 
award with development start. We assessed knowledge-based acquisition practices as not applicable 
in situations when programs submitted responses to our questionnaire that identified events or 
circumstances that made completion of a practice impractical. For example, we assessed the 
technology demonstration practice as not applicable in instances where a program did not have any 
critical technologies. 


Observation 2: Programs’ definitions of critical technologies varied. 
Most programs did not fully demonstrate mature technologies, and 
only about half completed system engineering reviews, before 
starting development. 

Establishing a consistent definition of the term “critical technology” is 
important for acquisition programs so they can correctly identify all critical 
technology elements. As we have previously reported, not doing so could 
result in an underrepresentation of the technical risk facing the program. 26 
GAO’s Technology Readiness Assessment Guide establishes criteria for 
identifying critical technology elements. 27 It states that a critical 
technology element is one that is new or novel, and necessary for a 
system to meet its anticipated operational performance requirements; or 
poses a major cost, schedule, or performance risk during design or 
demonstration. 

We observed that while the major acquisition programs in our review 
define the term “critical technology” differently from one another, these 
reported definitions frequently included several common elements. For 
instance, the three most common elements program officials included in 
their definitions were that 


26 GAO, Columbia Class Submarine: Immature Technologies Present Risks to Achieving 
Cost, Schedule, and Performance Goals , GAO-15-158 (Washington, D.C.: Dec. 21,2017). 

27 GAO, GAO Technology Readiness Assessment Guide: Best Practices for Evaluating the 
Readiness of Technology for Use in Acquisition Programs and Projects-Exposure Draft, 
GAO-16-410G (Washington, D.C.: Aug. 11,2016). 
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• the system being acquired depends on the technology element to 
meet requirements, 

• the technology is new or novel, and 

• the technology poses a major technological risk. 


Eighteen of 51 programs included these three elements in their definition 
of “critical technology.” 

According to knowledge-based best practices, programs should fully 
demonstrate critical technologies in a realistic environment and conduct a 
preliminary design review before starting system development. DOD and 
commercial technology development cases show the more mature 
technology is at the start of the program, the more likely the program will 
succeed in meeting its objectives. 28 Technologies that were included in a 
product development before they were mature later contributed to cost 
increases and schedule delays in those products. 29 

Table 10 presents the number of programs that had fully matured 
technologies when they began system development as well as the 
number of programs that had technologies approaching maturity at that 
time. The table presents the information for 42 current MDAPs that began 
development before or during our assessment period. 30 See appendix VIII 
for additional details on technology readiness levels. 


28 GAO-16-410G. 

29 lbid. 

30 Three of the current MDAPs we assessed entered the acquisition cycle post- 
development start. We therefore excluded these three programs from this analysis. 
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Table 10: Technology Readiness Levels (TRL) Reported by 42 Current Major Defense Acquisition Programs 


Thirty-eight programs that began development before our assessment period 

System prototype demonstration in an System/subsystem model or prototype 
operational environment (TRL 7) demonstration in a relevant environment 


(TRL 6) 


Number of systems 

Percentage of 
systems 

Number of 
systems 

Percentage of 
systems 

Met 

3 

8 percent 

24 

63 percent 

Not met 

30 

79 percent 

9 

24 percent 

Not applicable or information not 
available 

5 

13 percent 

5 

13 percent 


Four programs that began development during our assessment period 


System prototype demonstration in an System/subsystem model or prototype 
operational environment (TRL 7) demonstration in a relevant environment 

(TRL 6) 



Number of systems 

Percentage of 
systems 

Number of 
systems 

Percentage of 
systems 

Met 

0 

0 percent 

2 

50 percent 

Not met 

2 

50 percent 

0 

0 percent 

Not applicable or information not 
available 

2 

50 percent 

2 

50 percent 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


Note: A technology achieves TRL 7 when it has achieved form, fit, and function, and has been 
demonstrated in an operational environment. A technology that achieves TRL 6 is close to form, fit, 
and function, and has been demonstrated in a high-fidelity laboratory environment. Progressing from 
TRL 6 to TRL 7 is a significant step for a critical technology. While GAO’s best practices work has 
shown that a TRL 7 is the level of technology maturity that constitutes a low risk for starting 
development, DOD’s policy, permits development to start at TRL 6. DOD’s policy is based on a 
statute that generally prohibits a major defense acquisition program from receiving approval for 
development start until the milestone decision authority certifies—based on an independent review 
and technical risk assessment—that the technology in the program has been demonstrated in a 
relevant environment. 10 U.S.C. § 2366b(a)(2). 
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Impact of Immature Technologies 

In two extreme cases, the Family of Advanced 
Beyond Line-of-Sight Terminals (FAB-T) and 
the F-35 Joint Strike Fighter, the programs still 
had immature critical technologies that, at the 
start of development, were basically advanced 
breadboard demonstrators in a laboratory 
environment, which does not reach the 
maturity level associated with best practices 
for starting development. The current cost 
growth for the FAB-T and F-35 since their first 
full estimates is 31 percent and 51 percent, 
respectively. Each program also had 
significant schedule delays for declaring initial 
operational capabilities. The FAB-T IOC date 
slipped 96 months and the F-35 IOC date 
slipped 62 months. 

Source: GAO analysis of Department of Defense data. | GAO 
19-336SP 


According to knowledge-based practices, DOD policy, and statute, 
programs should conduct a preliminary design review (PDR) before 
beginning system development. 31 DOD policy also generally requires 
programs to complete formal system reviews in advance of beginning 
development. Further, best practices require programs to constrain their 
development periods to 6 years or less. 32 These reviews are intended to 
demonstrate that requirements are well understood and that the systems’ 
functional baselines can satisfy those requirements. 

Table 11 identifies the number of programs that held preliminary design 
reviews, key system reviews, and constrained development phases to 6 
years or less. The table presents information for 42 current MDAPs that 
began development before and during our assessment period. 33 


31 The purpose of a PDR is to assess the maturity of the preliminary design, supported by 
the results of requirements trades, prototyping, and critical technology demonstrations. 
The PDR establishes the allocated baseline (a definition of the configuration items making 
up a system, and then how system function and performance requirements are allocated 
across lower level configuration items) and confirms that the system under review is ready 
to proceed into detailed design. In detailed design, the program office will develop build-to 
drawings, software code-to documentation, and other fabrication documentation. By 
statute, a major defense acquisition program generally may not receive approval for 
development start until the milestone decision authority has received a preliminary design 
review, conducted a formal assessment of the preliminary design review, and certifies, 
based on that assessment, that the program has a high likelihood of accomplishing its 
intended mission. 10 U.S.C. § 2366b(a)(1). Under certain circumstances, this requirement 
may be waived. Id. § 2366b(d) 

32 GAO-1 8-360SP. 

33 Three of the current MDAPs we assessed entered the acquisition cycle post- 
development start. We therefore excluded these three programs from this analysis. 
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Table 11: Number of 42 Selected, Current Programs that Met Key Knowledge Practices Prior to Beginning Development 

Thirty-eight programs that began development before our 
assessment period 


Held a preliminary design 
review before starting 
development 

Held system reviews before 
starting development 

Constrain development phase 
to 6 years or less 

Yes 

19 

16 

19 

No 

17 

12 

8 

Not applicable or information 
not available 

2 

10 

11 

Four programs that began development during our 
assessment period 


Held a preliminary design 
review before starting 
development 

Held system reviews before 
starting development 

Constrain development phase 
to 6 years or less 

Yes 

0 

2 

2 

No 

1 

0 

0 

Not applicable or information 
not available 

3 

2 

2 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


• As in prior years, many programs in the 2018 portfolio began 
development without conducting a preliminary design review. DOD 
policy implementing related statute requires that PDR occur in 
advance of development start, unless the program’s milestone 
decision authority grants a waiver for this requirement. Of the 38 
programs that began development before our assessment period, 
only half conducted a preliminary design review prior to beginning 
system development. 34 

• Of the four programs that began development during our assessment 
period, two do not plan to conduct a preliminary design review, and 
one has yet to identify a date for its preliminary design review. Only 
one program—the Presidential Aircraft Recapitalization (PAR) — has 
conducted a PDR, but it occurred three months after beginning 
development, which is inconsistent with best practices. 

• Also, 12 programs did not hold system reviews in advance of starting 
development, as generally required by DOD policy. Three of those 


34 We did not evaluate why these programs did not hold a preliminary design review in 
advance of beginning development. In some cases, programs may have received a waiver 
from their milestone decision authority. 
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programs had cost growth of greater than 55 percent, while two other 
programs had modest cost growth of 10 percent or less. Two other 
programs, the Army’s JLTV program and the Air Force’s KC-46 
program, had cost reductions of 5 percent and 16 percent, 
respectively. 


Observation 3. Most programs did not fully demonstrate system 
design stability prior to conducting critical design reviews. 

We found that 35 programs held a knowledge point 2 event (critical 
design review) either before or during our assessment period. Table 12 
shows the number of those that satisfied the two associated key 
knowledge-based acquisition practices. Only two programs held a critical 
design review (or, for ships, began construction) during our assessment 
period. 


Table 12: Number of Programs Demonstrating Sufficient Design Knowledge Prior to Knowledge Point 2 

Thirty-three programs that held a critical design review before our assessment period 


Number of programs that tested an early 
system-level integrated prototype 

Number of programs that released at least 
90 percent of design drawings 

Yes 

4 

11 

No 

22 

17 

Not applicable or information not 
available 

7 

5 

Two programs that held a critical design review during our assessment period 


Number of programs that tested an early 
system-level integrated prototype 

Number of programs that released at least 
90 percent of design drawings 

Yes 

0 

2 

No 

1 

0 

Not applicable or information not 
available 

1 

0 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


• Seventeen of the 35 programs had not released 90 percent of their 
design drawings by the time of the critical design review, and five of 
those programs still had not released 90 percent of the design 
drawings by the time low-rate production began. Combined, those five 
programs experienced almost 27 percent average cost growth above 
their first full estimates. 

• The Navy’s CH-53K program did not test a system-level integrated 
prototype until 45 months after its critical design review. At the same 
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time, the program’s acquisition cycle time slipped 51 months and total 
acquisition costs grew by more than $10 billion—an increase of more 
than 55 percent. The Joint Precision Approach and Landing System 
system-level integrated prototype test was 85 months late and, 
consequently, the program delayed its initial operational capability 
declaration by 72 months and total acquisition costs grew by more 
than 69 percent. 

• The one non-ship program that conducted a critical design review 
during our assessment period, the B-2 Defensive Management 
System Modernization (DMS-M) program, did not test an early 
system-level prototype prior to its critical design review, but did so 7 
months afterward. The Navy’s Fleet Replenishment Oiler (T-AO 205) 
began lead-ship construction during our assessment period. Both the 
B-2 DMS-M and T-AO 205 programs delivered 90 percent of their 
design drawings to manufacturing in advance of their knowledge point 
2 events, and the programs experienced cost increases over their first 
full estimates of only 2.7 percent and 0.5 percent, respectively. Both 
programs also held system requirements reviews and system 
functional reviews prior to beginning development. 

• Of the six future acquisition programs we assessed, only one, the B- 
52 Radar Modernization Program, identified a date for testing an early 
system-level integrated prototype. However, this event is scheduled 
more than two years after the program’s critical design review. 


Observation 4. Only one program we reviewed reported that its 
manufacturing processes were in statistical control prior to starting 
production. 

Programs that held production decisions during our assessment period 
demonstrated higher levels of production-related knowledge than 
programs that started production in advance of our assessment period. In 
fact, all three programs that began production during our assessment 
period satisfied the knowledge requirements for two of the key practices 
associated with production—testing a production-representative prototype 
in its intended environment and demonstrating critical processes on a 
pilot production line. However, none of those three programs 
demonstrated the knowledge associated with a third key production- 
related practice—demonstrating manufacturing processes are in 
statistical control. 

Table 13 identifies the number of programs that satisfied those three key 
production-related practices. The table presents the information for 
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programs that both began production before our assessment period and 
those that began production during our assessment period. 


Table 13: Program Completion of Key Production-Related Knowledge Practices 

Fifteen programs that began production before our assessment period 


Test a production- 
representative prototype in its 
intended environment 

Demonstrate critical 
processes on a pilot 
production line 

Demonstrate critical 
manufacturing processes are 
in statistical control 

Yes 

7 

8 

1 

No 

8 

5 

13 

Not applicable 

0 

2 

1 

Three programs that began production during our assessment period 


Test a production- 
representative prototype in its 
intended environment 

Demonstrate critical 
processes on a pilot 
production line 

Demonstrate critical 
manufacturing processes are 
in statistical control 

Yes 

3 

3 

0 

No 

0 

0 

3 

Not applicable 

0 

0 

0 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


Production Knowledge 

Of the 15 programs that began production in 
advance of our assessment period, only one, 
the Infrared Search and Track, reported 
compliance with all three production-related 
knowledge practices contained in table 13. 
This program also satisfied most, but not all, 
other knowledge practices throughout its 
acquisition cycle and experienced virtually no 
cost changes since its first full estimate. 

The Common Infrared Countermeasure, Joint 
Air to Ground Missile, and the Amphibious 
Combat Vehicle all began production during 
our assessment period. Each program 
satisfied two of the three production-related 
knowledge practices noted above. These 
programs also completed most, but not all, of 
the knowledge practices throughout their 
lifecycles. The Amphibious Combat Vehicle 
reported a cost reduction from its first full 
estimate, while the Joint Air to Ground Missile 
and the Common Infrared Countermeasure 
reported modest cost growth since first full 
estimate of 2.3 percent and 0.1 percent, 
respectively. 

Source: GAO analysis of Department of Defense data. | GAO 
19-336SP 


Observation 5. Future programs provided mixed signals on their 
planned implementation of knowledge-based acquisition practices. 

The six future programs we surveyed reported that they plan to 
incorporate some, but not all, of selected knowledge-based acquisition 
practices associated with development start. A few of these future 
programs, which remain in their early planning stages, have yet to 
establish dates for events that serve as key indicators for compliance. 
Table 14 presents the number of future MDAPs that plan to meet three 
key knowledge practices associated with beginning system development. 
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Table 14: Planned Implementation of Selected Knowledge-Based Acquisition Practices at Development Start among Six 

Future Major Defense Acquisition Programs 


Plan to demonstrate all 
critical technologies in a 
realistic environment 

Plan to complete all system 
engineering reviews 

Plan for a development phase 
of less than six years 

Yes 


0 

2 

2 

No 


0 

0 

0 

To be determined 


3 

1 

1 

Information not available 


3 

0 

0 

Not applicable 


0 

2 

2 

Not applicable—ship 


0 

1 

1 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


• Of the six future acquisition programs we surveyed, three have 
identified specific critical technologies. Our prior work has shown that 
programs beginning development with immature technologies run a 
greater risk of cost growth and schedule delays than programs that 
satisfy the knowledge practices. 

• Of the three programs that identified critical technologies, the Air 
Force’s Weather System Follow-on program has the nearest planned 
system development start date (March 2019). The program has 
identified eight critical technologies, three of which the program 
reported as being equivalent to a high-fidelity breadboard 
demonstration in a laboratory environment, which does not reach the 
maturity level associated with best practices for starting development. 

• Four of the six plan to complete PDR prior to starting development. 
However, other schedule dates were either not yet available or not 
applicable, as is the case with Weather System Follow-on, which will 
not have a production decision. 


Observation 6. Certain knowledge-based acquisition practices 
correspond with better program outcomes. 

For the second consecutive year, we conducted an exploratory statistical 
correlation analysis to determine whether a statistically significant link 
exists between non-ship programs’ cost and schedule performance and 
their implementation of knowledge-based acquisition practices. We found 
that, generally, programs that complete certain knowledge practices have 
better cost and schedule outcomes than programs that do not implement 
those practices. 
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This year, we analyzed 17 programs—an increase of two programs as 
compared to our 2018 analysis—that have completed system 
development, held a critical design review, and started production (i.e., 
completed knowledge points 1 through 3). 35 We observed that MDAPs 
that completed preliminary design review before system development 
start had an estimated cost savings of 0.5 percent compared to an 
estimated 65.6 percent cost increase for MDAPs that did not. Further, we 
observed that this best practice is also associated with shorter schedules 
to deliver capabilities to the warfighter. Namely, our analysis found a 33.5 
percent increase in schedule when programs implement this practice as 
compared to an 80.5 percent increase when programs do not implement 
it. 

Programs with technologies that were at least approaching maturity also 
realized, on average, improved cost and schedule performance. 

Programs implementing this practice realized an average cost reduction 
of 5.4 percent, while those not implementing the practice had cost growth 
of 24.0 percent. Programs implementing this practice also realized less 
growth in acquisition cycle times, 21.5 percent, versus 42.8 percent for 
those programs that did not implement this practice. 

Further, in our 2018 exploratory analysis, we identified a third knowledge 
practice—delivery of 90 percent of design drawings to manufacturing prior 
to critical design review—as statistically significant in producing better 
cost and schedule outcomes in programs. 36 This year’s analysis did not 
revalidate our 2018 analysis, but we plan to continue assessing this 
measure in future statistical correlation analyses we conduct. 

These results support our work and offer continued validation that MDAPs 
that follow these particular knowledge-based practices achieve lower cost 
increases and less schedule growth than those programs that do not. Our 
prior work demonstrates that completion of all of the knowledge-based 
practices by the time programs reach their knowledge points underpins a 


35 We conducted a means test comparing averages across systems that did and did not 
complete knowledge-based acquisition practices using a 90 percent confidence interval. 
See appendix II for additional details. 

36 GAO-1 8-360SP. 
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sound business case that positions programs to better meet their cost 
and schedule goals. 37 

As with last year’s annual assessment, the small sample size and the 
unique characteristics of the 17 MDAPs in the data set limit our analysis. 
As more programs within our scope enter production, we will be able to 
offer a more statistically significant analysis of differences between 
MDAPs that completed or did not complete specific knowledge-based 
practices. 

Observation 7. Programs continue to identify software as a risk area 
and many do not plan to deliver a baseline software capability prior 
to production start. 

Current weapon systems are dependent upon software for providing 
warfighter capabilities, as we have reported in prior work. 38 Yet despite 
this reality, many programs are not tracking software development costs, 
are underestimating the difficulty of software development efforts, or are 
not utilizing key software metrics. 

According to a 2018 Defense Science Board report, “software is a crucial 
and growing part of weapons systems and the national security 
mission.” 39 Software is increasingly driving the amount and types of 
capability that weapon systems provide to the warfighter, and software is 
an item that often represents considerable risk for weapons acquisition 
programs. In order to develop that software, programs can choose from a 
number of different approaches, and there is no single approach that is 
best for every situation. Table 15 presents a few of the approaches that 
are more frequently utilized among DOD acquisition programs and their 
corresponding definitions. 


37 ln general, a business case is a justification for a proposed project or undertaking. We 
have reported that a sound business case for successful defense acquisition programs 
contains key elements, including firm requirements, mature technologies, a knowledge- 
based acquisition strategy, a realistic cost estimate, and sufficient funding. 

38 GAO, Weapon Systems Cybersecurity: DOD Just Beginning to Grapple with Scale of 
Vulnerabilities, GAO-19-128 (Washington, D.C.: Oct. 9, 2018). 

39 DOD, Defense Science Board: Design and Acquisition of Software for Defense Systems 
(Washington, D.C.: Feb. 14, 2018). 
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Table 15: Software Development Models Employed by 51 Selected Current and Future Major Defense Acquisition Programs 

Software development life 
cycle model 

Description 

Waterfall 

This model relies on strict phases, and each phase needs to be completed before going to the next 
phase. The phases include requirements definition, design, execution, testing, and release. Each 
phase relies on information from the previous phase. This model is a linear sequential flow in which 
progress is seen as flowing steadily downwards (like a waterfall) through the phases of software 
implementation. 

Incremental 

This model sets high level requirements early in the effort, and functionality is delivered in stages. 
Multiple increments delivery part of the overall required program capability. Several builds and 
deployments are typically necessary to satisfy approved requirements. 

Spiral 

This model takes ideas from the incremental model and its repetition but also combines the structured 
and systematic development of the waterfall model with a heavy emphasis on risk analysis. The 
project passes through four phases (identification, design, build, evaluation and risk analysis) over 
and over in a “spiral” until completed, allowing for multiple rounds of refinement. 

Agile 

This model breaks a product into components where on each cycle or iteration, a working model of a 
component is delivered. The model produces ongoing releases, each time adding small changes to 
the previous release. During each iteration, as the product is being built, it is also tested to ensure that 
at the end of the iteration the product is shippable. The Agile model emphasizes collaboration, as the 
customers, developers, and testers work together throughout the project. 

Hybrid 

This approach is a combination of two or more different methodologies or systems to create a new 
model. 


Source: GAO analysis of Department of Defense and software industry documentation. | GAO-19-336SP 


Programs we surveyed reported using a variety of these approaches, and 
many programs use multiple software development approaches when a 
single approach is not the best option for the capabilities being 
developed. Figure 17 shows the various software development models 
that selected current and future MDAPs have employed. 
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Figure 17: Distribution of Software Development Models among 51 Selected Current 
and Future Major Defense Acquisition Programs 

Number of programs per department 
18 

16 
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12 
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8 
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Agile Waterfall Incremental Spiral Hybrid Other 

Software development models 

DOD 
Air Force 
Navy 
Army 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 

Note: “Other” category represents any software development approach that programs identified to us 
that does not fall into one of the five DOD-identified models. In addition, some programs reported 
using multiple approaches. As a result, the columns in the figure do not sum to 51. 





• Among the 51 programs we surveyed, the most frequently utilized 
software development model was the incremental approach, with 17 
programs reporting its use. The Air Force’s Combat Rescue 
Helicopter is an example of a program using multiple software models 
for various subsystems, and one of those models is the incremental 
approach. The spiral model is the least utilized. Only one program, the 
Navy’s DDG 1000, reported using this approach. 

• Some program officials reported using “other” software development 
methods, for a variety of reasons. The Navy’s Virginia Class 
Submarine (SSN 774) initially used a waterfall-like approach to deliver 
required software. However, program officials reported that the SSN 
774 program no longer employs that approach because software 
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deliveries are complete and the Navy is modifying existing software to 
accommodate revised ship characteristics. 

• Eleven Navy programs reported utilizing the agile software 

development method, the most among any military department. Only 
two Air Force programs and one Army program reported using the 
agile software method. 


Table 16 identifies the number of current major programs we surveyed 
that identified software as a risk at any point during the program’s history, 
as well as the number of programs that satisfied key software-related 
practices we identified in prior work. 40 


Table 16: Software Risk Identification and Development Practices among 45 Selected Current Major Defense Acquisition 
Programs 


Software identified as 
a risk area 

Baseline capability 
completed before 
production 

Baseline capability 
completed before initial 
operational capability 

Software development 
costs tracked 

Yes 

33 

16 

27 

15 

No 

11 

15 

4 

30 

Do Not Know 

1 

0 

0 

0 

Information Not Available 

0 

14 

14 

0 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


• Programs cited three main reasons why they identified software as a 
risk area: 

• completing the software effort in order to conduct developmental 
testing successfully, 

• changes for cybersecurity needs leading to additional software 
development efforts, and 

• completing the originally planned software development effort has 
proven more difficult than expected. 

• Software development has similar phases to that of hardware and—in 
the case of new systems—occurs in parallel with hardware 
development until software and hardware components are integrated. 


40 GAO, Defense Acquisitions: Stronger Management Practices Are Needed to Improve 
DOD’s Software-Intensive Weapon Acquisitions, GAO-04-393 (Washington, D.C.: Mar. 1, 
2004). 
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According to DOD policy, major contracts and subcontracts for 
contractors developing or producing software elements for MDAPs 
are subject to software resource data reporting requirements if the 
software effort is projected to be greater than $20 million. 41 However, 
when programs do not track the cost of software development 
separately from the overall development effort, DOD’s ability to 
impose the reporting requirement is constrained. 

• We have previously found that programs that defer completion of 
software development activities to post-production phases—such as 
after initial operational capability or initial operational test and 
evaluation—risk delivering systems to the warfighter that do not meet 
their minimum performance requirements. 42 

Figure 18 shows the points in the acquisition process when the 45 current 
MDAPs we surveyed completed, or plan to complete, software 
development. 


41 Specifically, all major contracts and subcontracts, regardless of contract type, for 
contractors developing or producing software elements within current and future MDAPs, 
as well as other certain acquisitions, such as those of certain automated information 
systems, for any software development element with a projected software effort greater 
than $20 million (then-year dollars) are required to complete a Software Resources Data 
Report. DOD Instruction 5000.02, Operation of the Defense Acquisition System (Jan. 
2015) [incorporating change 4 (Aug. 2018)] (“DOD Instruction 5000.02”). 

42 GAO-1 8-360SP. 
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Figure 18: Plans for Completing Software Development among 45 Selected Current Major Defense Acquisition Programs 
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Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


DOD acquisition policy identifies six acquisition models that serve as 
examples of defense program structures tailored to the type of product 
being acquired or to the need for accelerated acquisition. 43 Among other 
things, these models emphasize software development to varying 
degrees. The policy directs acquisition programs to use these models as 
a starting point in structuring a program to acquire a specific product. 
Table 17 identifies these six models in more detail. 


43 DOD Instruction 5000.02. 
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Table 17: Department of Defense’s (DOD) Six Acquisition Models and Their Basic Characteristics 

Acquisition Model 

Basic Characteristics 

Hardware intensive program 

This is the classic model for a major weapon program and has existed in some form in all previous 
editions of DOD’s acquisition policy. 

Defense unique software 
intensive program 

Uses a series of planned “software builds”—testable, integrated subsets of the overall capability— 
which together with clearly defined decision criteria, ensure adequate progress is being made before 
the program commits to subsequent builds. 

Incrementally deployed 
software intensive program 

Deploys full software capability in multiple increments as new capability is developed and delivered, 
nominally in 1- to 2-year cycles. 

Accelerated acquisition 
program 

Prioritizes schedule considerations over cost and technical risk considerations by compressing or 
eliminating phases of the acquisition process and accepting the potential for inefficiencies in order to 
achieve a deployed capability on a compressed schedule. 

Hybrid program A (hardware 
dominant) 

Combines a basic structure hardware development program with a simultaneous, software intensive 
development effort. Design, fabrication, and testing of physical hardware prototypes may determine 
the program’s overall schedule, decision points, and milestones, but software development will often 
dictate the pace of program execution and must be tightly integrated and coordinated with hardware 
development decision points. 

Hybrid program B (software 
dominant) 

Centers on a software intensive product development that can include a mix of incrementally 
deployed software products or releases that include intermediate software builds. 

Source: GAO analysis of DOD Instruction 5000.02.| GA0-19-336SP 


We asked each program we surveyed to identify the specific model that 
was most applicable to their program. Figure 19 identifies the distribution 
of development models among 45 major defense programs. 
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Figure 19: Plans for Utilizing System Development Models among 45 Current Major 
Defense Acquisition Programs 
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Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


We compared the acquisition models and software development plans 
that programs reported and observed the following: 

• Of the 15 programs that identified Hybrid Program A (hardware 
dependent) as their model, 13 reported to us that they had identified 
software as a risk area for the program. Further, nine of these 15 
programs tracked the cost associated with their software development 
efforts, while six of those 15 programs reported that a baseline 
software capability would be available in time for the start of 
production. 

• Of the 14 programs that identified Hardware Intensive Program as 
their model, half the programs either reported that they did not know 
or do not track the estimated total acquisition costs specifically for 
software. At the same time, six of these programs reported software 
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as a risk area, and only four reported that a baseline software 
capability would be ready for a production decision. 

Observation 8. Many programs we surveyed declared, or plan to 
declare, initial operational capability without conducting initial 
operational test and evaluation. 

Initial operational capability occurs when a unit or organization has been 
equipped and trained and is determined to be capable of conducting 
operations with a newly fielded system. However, each individual 
program declares its initial operational capability date, and programs do 
not apply consistent standards when determining that date. Initial 
operational test and evaluation is a separate event that is intended to 
evaluate a system’s effectiveness and suitability under realistic 
operational conditions before a program makes a full-rate production 
decision. We have observed programs declaring initial operational 
capability on the basis of full, partial, or no initial operational test and 
evaluation. Programs that declare initial capability before completing 
initial operational test and evaluation risk fielding systems to warfighters 
that are not operationally effective or suitable for the missions they will be 
tasked to perform. Figure 20 details our analysis of the relationship 
between initial operational capability and initial operational test and 
evaluation in 45 current MDAPs. 


Figure 20: Plans for Declaring Initial Operational Capability among 45 Selected Current Major Defense Acquisition Programs 
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Assessments of 
Individual Programs 


This section contains (1) 51 assessments of individual weapon programs 
and (2) four summary analyses—each segmented by military department. 
Each assessment presents data on the extent to which programs are 
following a knowledge-based acquisition approach to product 
development. 44 Each military department’s summary analysis page 
presents aggregated information about selected programs’ acquisition 
phases, current estimated funding needs, knowledge attained, cost and 
schedule performance, oversight authorities, software characteristics, and 
competition in contracting. To develop the individual program 
assessments, we collected program information using a web-based 
questionnaire, obtained and reviewed agency documents, and 
interviewed program officials. 

For 39 programs, all of which were in development or early production, 
we produced two-page assessments discussing the technology, design, 
and manufacturing knowledge obtained, as well as other program issues. 
Each two-page assessment also contains a comparison of total 
acquisition cost from the first full estimate for the program to the current 
estimate. The first full estimate is generally the cost estimate established 
at development start; however, for a couple of programs that did not have 
such an estimate, we used the estimate at production start. For 
shipbuilding programs, we used their planning estimates if those 
estimates were available. For programs that began as non-MDAPs, we 
used the first full estimate available. See figure 21 for an illustration of the 
layout of each two-page assessment. 


44 The assessments also contain basic information about the program, including the prime 
contractor(s) and contract type(s). We abbreviated the following contract types in the 
individual assessments: cost-plus-award-fee (CPAF), cost-plus-fixed-fee (CPFF), cost- 
plus-incentive-fee (CPIF), firm-fixed-price (FFP), fixed-price incentive (FPI), and indefinite- 
delivery / indefinite-quantity (IDIQ). 
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Figure 21: Illustration of Program Two-Page Assessment 


00 


o 



©—► 


Common Name. B-2 DMS-M 


B-2 Defensive Management System 
Modernization (B-2 DMS-M) 

Tho Air Force's B 2 DMS M program plans to upgrade the aircraft's 
1980s-ora dofonslvo management system to a more capable system 
This system detects and locates enemy radar systems to provide threat 
warnings and avoidance information This upgrade is expected to improve 
the system's frequency coverage and sensitivity, update pilot displays, and 
enhance in-flight rerouting capabilities. It will improve the reliability and 
maintainability of the DMS system and the B-2 s readiness. 
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Lead Component: Air Force 

B-2 DMS-M Program ^ f 
Technology Maturity and Design Stability 

The B-2 DMS-M program has fully matured its critical 
technologies and stabilized its design. The program 
entered system development in March 2016 with four 
critical technologies approaching maturity. Since then, 
Northrop Grumman proposed, and the Air Force accepted, 
an alternative system that provides additional capabilities. 
The alternative system is leveraged from other programs 
and has the same four critical technologies. Because the 
Air Force has tested the major subsystems on an existing 
program, it now considers its critical technologies mature. 

The Air Force's decision to pursue an alternative system 
led to some initial program delays. For example, although 
the Air Force held a preliminary design review before 
entering system development, it completed a second 
preliminary design review for (he alternative system in 
March 2018. The program also held its critical design 
review in November 2018. nearly 2 years later than 
originally planned. Consistent with acquisition best 
practices, the program released more than 90 percent of 
its design drawings by the critical design review. However, 
the program did not test a system-level integrated 
prototype—an approach Inconsistent with acquisition best 
practices. This decision puts the program at risk for costly 
design changes following system-level integration testing 
if the program identifies deficiencies. 

The program continues to face delays and Increased 
integration risk in its software development, a critical 
factor tor achieving required B-2 DMS-M capabilities. 

The contractor is utilizing an Agile software development 
process for the first time on this program and has taken 
longer than expected to achieve the development 
efficiency to complete the functionality of a software 
block, known as PD 7.1. The program requires 
certification of software block PD 7.1 functionality to 
support developmental flight testing. Program officials 
stated that over the last year the contractor has made 
significant progress in addressing software development 
risk by increasing the number of software developers and 
expanding overtime for existing staff. However, software 
version PD 7.1 may still be at risk of not being certified 
with full functionality by June 2019. when the program has 
scheduled developmental flight tests. Lack of software 
certification could delay these tests. Program officials 
continue to monitor the effectiveness of the contractor's 
mitigation plans and possible schedule delays. 

Other Program Issues 
Program officals report that, in May 2017. due to the 
selection of the alternative DMS system, the program 
office and contractor agreed to an undefimtized contract 
action that changed the development contract from a cost- 


Name; B-2 DMS-M 


type contract to a firm-fixed-price contract. The program 
office stated that the undefinitized contract action included 
a not-to-exceed amount of $741 million until the contract 
scope and price could be definltlzed. Because there were 
issues with the contractor delivering a complete proposal, 
the program has delayed definitization. The program office 
reports approximately 50 percent of the $741 million not- 
to-exceed amount has been obligated, and officials do not 
expect to definitize the contract action until the spring of 
2019. nearly 2 years after award. 

As program officials began negotiations for using the 
alternative DMS system, they recognized they could not 
execute the related development work within the existing 
Air Force budget. As a result, the Air Force conducted a 
cost-versus-capabillty trade analysis of the program and 
ultimately decided on a course of action that continued 
with the alternative system but required hardware changes 
on the aircraft In June 2018. the Air Force released an 
updated service cost position to reflect the latest acquisition 
approach, which identified a 13 percent unit cost increase 
over the program's acquisition baseline. This increase was 
largely due to hardware changes and integration updates 
as a result of the increased capabilities of the alternative 
DMS system. 

Program Office Comments 
We provided a draft of this assessment to the program 
office for review and comment The program office provided 
technical comments, which we incorporated where 
appropriate, In reviewing the draft, program officials noted 
that the revised program strategy leverages development 
from other Air Force programs, implements additional 
capability to improve weapon system survivability, and 
provides risk reduction for other programs 

In addition, program officials stated the oost-versus- 

capability analysis ensured that while hardware J _ 

requirements were decreased in order to reduce > 

procurement costs, the program will still meet performance 
requirements.They also stated that the service cost 
position now estimates a 12.5 percent increase in program 
acquisition unit cost as well as the potential for delays .This 
estimate includes the increased capabilities and associated 
risk reduction, and is not constrained to the not-to-exceed 


Program officials also noted that design efforts are 
complete as presented at the critical design review They 
said that DMS-M installation into the B-2 flight test aircraft 
is underway. According to the officials, the critical path to 
beginning flight test is completion of software development 
and PD 7.1 certification efforts. The program office stated 
that it continues to work closely with the prime contractor to 
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© Illustration or photo of system 

© Program description 

0 Schedule timeline identifying key dates for the program, including the 
start of development, major decision reviews, production decisions, 
and planned operational capability 

© Program Essentials Programmatic information, including milestone 
decision authority, program office location, prime contractor, contract 
type, software development approach, software development 
approach, and next major milestone 

@ Program Performance Cost and schedule baseline estimates and 
the latest estimate provided 


© 

G Remaining Funding Requirements Development and production 
funding and quantity requirements from fiscal year 2019 through 
completion 

® Attainment of Product Knowledge Depiction of selected 
knowledge-based practices and the program’s progress in 
attaining that knowledge 

© Assessment of program’s technology, design, and production 
maturity, as well as other program issues 

O Program Office Comments General comments provided by the 
cognizant program office 


Source: GAO. | GAO-19-336SP 

In addition, we produced one-page assessments on the current status of 
12 programs, which include (1) six future MDAPs; (2) one MDAP that was 
in development, but released its cost baseline very late in our review, and 
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(3) five MDAPs that were well into production, but planned to introduce 
new increments of capability. See figure 22 for an illustration of the layout 
of each one-page assessment. 


Figure 22: Illustration of Program One-Page Assessment 
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Current Status 


The FFG(X) program continues conceptual design work ahead of planned 
award of a lead ship detail design and construction contract in September 
2020. In May 2017, the Navy revised its plans for a new frigate derived 
from minor modifications of an LCS design. The current plan is to select a 
design and shipbuilder through full and open competition to provide a more 
lethal and survivable small surface combatant. 

As stated in the FFG(X) acquisition strategy, the Navy awarded conceptual 
design contracts in February 2018 for development of five designs based 
on ships already demonstrated at sea. The tailoring plan indicates the 
program will minimize technology development by relying on government 
furnished equipment from other programs or known contractor furnished 
equipment. 

In November 2018, the program received approval to tailor its acquisition 
documentation to support development start in February 2020. This 
included waivers for several requirements, such as an analysis of 
alternatives and an affordability analysis for the total program life cycle. 
FFG(X) also received approval to tailor reviews to validate system 
specifications and the release of the request for proposals for the detail 
design and construction contract. 

Following the development start decision, the program office will complete 
its evaluation of FFG(X) proposals and plans to award a contract in 
September 2020 to a single shipbuilder for detail design and construction of 
the lead frigate, with options for nine additional ships. 

Attainment of Technology Maturation Knowledge 

As of January 2019 
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Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment The program office did not have any comments. 
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For 45 of our one- and two-page assessments, we used scorecards to 
depict the extent of knowledge that a program has gained. 45 These 
scorecards display key knowledge-based acquisition practices that should 
be implemented by certain points in the acquisition process. In our prior 
and current work, we found that the more knowledge a program has 
attained by each of these key points, the more likely the weapon system 
will be delivered within its estimated cost and schedule. In a current 
MDAP, knowledge deficits signal that the program is proceeding without 
sufficient knowledge about its technologies, design, or manufacturing 
processes, and faces unresolved risks that could lead to cost growth and 
schedule delays. 

For each scorecard, we used a closed circle to denote a knowledge- 
based practice the program implemented. We used an open circle to 
denote a knowledge-based practice the program did not, or has not yet, 
implemented. For future MDAPs only, we used a partially closed circle to 
denote a knowledge-based practice that the program reported it plans to 
implement. If the program did not provide us with enough information to 
make a determination, we showed this with a dashed line. We also 
marked as “NA” any scorecard field that corresponded with a knowledge- 
based practice that was not applicable to the program. A knowledge- 
based practice may not be applicable to a particular program if it has not 
yet reached the point in the acquisition cycle when the practice should be 
implemented, or if the particular practice is not relevant to the program. 
For shipbuilding programs, we assessed different key points in the 
acquisition cycle and applicable knowledge-based practices, which were 
informed by our prior work. 46 Appendix II provides additional detail on our 
scorecard methodology. Figures 23 and 24 provide examples of the 
knowledge scorecards we used in our assessments. 


45 We did not use scorecards in five of our one-page assessments of MDAPs that were 
well into production, but planned to introduce new increments of capability, because our 
metrics on knowledge attainment were incongruent with the acquisition strategies these 
programs employed. We did not use a scorecard in a sixth one-page assessment—for 
VC-25B Presidential Aircraft Recapitalization—which released a baseline late in our 
review. 

46 GAO-09-322. 
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Figure 23: Examples of Knowledge Scorecards on Two-Page Assessments 
Program in production Shipbuilding program 


Attainment of Product Knowledge 

As of January 2019 Status at 

Resources and requirements match 

• Demonstrate all critical technologies are very close to 
final form, fit, and function within a relevant environment | 

• Demonstrate all critical technologies in form, fit and 
function within a realistic environment 


• Complete a system-level preliminary design review 

• 


Product design is stable 

Design 

review 


• Release at least 90 percent of design drawings 

• 

• 

• Test a system-level integrated prototype 

o 

o 

Manufacturing processes are mature 

Production 

start 

• Demonstrate Manufacturing Readiness Level of at least 

9 or critical processes are in statistical control 

NA 

NA 

• Demonstrate critical processes on a pilot production line 

NA 

NA 

• Test a production-representative prototype in its 
intended environment 

NA 

NA 


Knowledge attained 


Information not available 
M Knowledge not attained |££J Not applicable 


Attainment of Product Knowledge 

As of January 2019 

Resources and requirements match 


Status at 

Detail design 
contract 
award 


Current 

status 


• Demonstrate all critical technologies are very close to 
final form, fit, and function within a relevant environment 

o 

• 

• Demonstrate all critical technologies in form, fit and 
function within a realistic environment 

o 

o 

• Complete a system-level preliminary design review 

o 

• 

Product design is stable 

Fabrication 

start 


• Complete basic and functional design to include 

100 percent of 3D product modeling 

o 

1 


I! Knowledge attained Information not available 

E] Knowledge not attained DQ Not applicable 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


Figure 24: Example of Knowledge Scorecard on One-Page Assessments 
Attainment of Technology Development Knowledge 


As of January 2019 


Conduct competitive prototyping 


Validate requirements 



Complete technology readiness assessment 


Complete preliminary design review 



^Knowledge i 


i attained, 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


2J Knowledge planned, Knowledge 


not attained, 


Information not available, 


3 Not applicable 
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ARMY PROGRAM 
ASSESSMENTS 


GAO Assessed 7 of the 
Army’s 23 Current Major 
Defense Acquisition 
Programs 


The 7 Current Programs GAO 
Assessed Represent 41 Percent of 
the Army’s $189 Billion Estimated 
Total Acquisition Costs 

(Fiscal Year 2019 dollars in billions) 


Current program — ^ 

Current 
programs 

(Not assessed) 

■ We also assessed two future major defense acquisition programs. We did not assess programs that 
had yet to enter technology development or had already progressed into full-rate production. 




Most Current Programs GAO Assessed 
Have Either Cost Growth, Schedule 
Delays, or Both 

Program 

complete 


Total Acquisition Cost Performance 
in Current Programs Varied Among 
Knowledge Points (KP) 

(Fiscal Year 2019 dollars in billions) 


Most Current Programs Did Not Attain 
Knowledge at Key Points 



Programs in 
production 



Programs in system 
development 

AMPV 


KP1 


$3.5 

$2.3 


-$6.7 

KP1 to 
KP2 


KP2 to 
KP3 


Post - 
KP3 


KP2 

t , 

KP3 


A cost decrease due to a significant quantity 
decrease in one current program—HMS—accounted 
for most of the cost decrease between KP1 and KP2. 


Oof 7 

current 

programs 

attained 

knowledge 


1 of 7 

current 

programs 

attained 

knowledge 


Oof 5 

current 

programs 

attained 

knowledge 


CIRCM 




Average Acquisition Cycle Time of Current Programs Increased by 
16 Months from First Full Estimate (FFE) to Current Estimate 


0 5 10 15 20 25 30 35 40 

Program age 

• Cost growth of more than 15 percent and/or 
schedule delays of more than 6 months 

• Cost growth of 15 percent or less and/or 
schedule delays of 6 months or less 

• Cost growth of 0 percent or less and 
schedule delays of 0 months or less 
Note: Bubble size is based on each program's currently 
estimated future acquisition funding needed. 


Programs' 
average FFE 
cycle time 


Programs' average 
cycle time change from 
FFE to current estimate 

Number of months 



■ One current program—ITEP—entered system 
development after December 2017 and therefore 
is excluded from our cost, schedule, knowledge 
attainment, and contract analysis. 


Cost and schedule analysis is primarily based on estimates from December 2017 selected acquisition reports. 
This information may differ from the amounts reported in the individual assessments, which are based on more 
recent program estimates in some cases. Please see appendix II for details. 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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ARMY PROGRAM 
ASSESSMENTS 


Most Current and Future Army 
Programs Report that Less than 20 
Percent of Planned Acquisition Costs 
are Specifically for Software 


The Army was the Milestone Decision Authority in 2017 for 
9 of its Current and Future Programs in GAO’s Review 

Army programs 


Percentage of planned acquisition costs 


0 - 20 % 


21-40% 


USD(A&S) 

programs 




The milestone decision authority 
for AMPV, CIRCM, HMS, JAGM, 
and PrSM changed from the 
Under Secretary of Defense for 
Acquisition and Sustainment 
(USD(A&S)) to the Army in 2017. 


NA/not 

tracked 


The Army Received One Offer on Solicitations for Just Over Half of its 
Currently Reported Major Contracts for Development and Procurement 


0 5 10 15 20 

Number of programs 


Most Current Army Programs Report 
that Less than 20 Percent of Work 
Tasks are Software-Related 


Percentage of work tasks 


0 - 20 % ■ 

21-40%l 

0 5 10 15 20 

Number of programs 


Number of contracts 



C3I 


Combat 

Ground Helicopter 

Missile 

Other 

Vehicle 

Combat 




System type 


2 or more offers 



offer 


C3I is short for command, 
control, communications and 
intelligence 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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Army Program Assessments* 

Page 

number 

2-page assessments 

Armored Multi-Purpose Vehicle (AMPV) 

75 

CH-47F Modernized Cargo Helicopter (CH-47F Block II) 

77 

Common Infrared Countermeasure (CIRCM) 

79 

Handheld, Manpack, and Small Form Fit Radios (HMS) 

81 

Integrated Air and Missile Defense (IAMD) 

83 

Improved Turbine Engine Program (ITEP) 

85 

Joint Air-to-Ground Missile (JAGM) 

87 

Joint Light Tactical Vehicle (JLTV) 

89 

1-page assessments 

Indirect Fire Protection Capability Increment 2 - Intercept Block 1 (IFPC Inc 2-1 Block 1) 

91 

Precision Strike Missile (PrSM) 

92 


a We abbreviate the following contract types in the individual assessments: cost-plus-award-fee (CPAF), cost-plus-fixed-fee (CPFF), cost-plus-incentive- 
fee (CPIF), firm-fixed-price (FFP), fixed-price incentive (FPI), and indefinite-delivery / indefinite-quantity (IDIQ). 
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Lead Component: Army 


Common Name: AMPV 



Armored Multi-Purpose Vehicle (AMPV) 

The Army's AMPV is the replacement to the Ml 13 family of vehicles at 
the brigade level and below. The AMPV will replace the Ml 13 in five 
mission roles: general purpose, medical evacuation, medical treatment, 
mortar carrier, and mission command. The Army determined that 
development of the AMPV is necessary due to mobility, survivability, 
and force protection deficiencies identified with the Ml 13, as well as 
space, weight, power, and cooling limitations that prevent the 
incorporation of future technologies. 


Q. 

LU 

O 

Z 

o 

o 




LU 

CL 

2 O 

LU -I 

H UJ 
CO > 
> LU 
CO Q 


12/14 

Development 

start 


06/16 

Design 

review 




■®—®- 


■®. 




o 

3 

Q 

O 

CL 


01/19 01/19 02/21 06/21 10/21 03/22 

Low-rate GAO Start End Full-rate Initial 

decision review operational operational decision capability 

test test 


Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Army 


First full estimate 

Latest 

Percentage 

Program office: Warren, Ml 


(05/2015) 

(06/2018) 

change 

Prime contractor: BAE Systems 

Contract type: CPIF (development) 

Development 

$1,050.90 

$1,051.30 

0.0% 

Software development approach: 

Procurement 

10,354.8 

$10,466.20 

+1.1% 

Incremental 

Unit cost 

$3.89 

$3.95 

+1.8% 

Next major milestone: Start operational 
testing (February 2021) 

Acquisition cycle 
time (months) 

87 

87 

0.0% 


Total quantities 

2,936 

2,936 

0.0% 


Total quantities comprise 39 development quantities and 2,897 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$292.46 ^ $758.80 


o 


Procurement 



107 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
| To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Lead Component: Army 

AMPV Program 


Common Name: AMPV 


Technology Maturity and Design Stability 

The AMPV program has matured its critical 
technologies and stabilized its design. The program 
entered system development in December 2014 with its 
critical technologies deemed fully mature by an 
independent review team. Program officials noted that 
although the AMPV uses a new hull design, a majority 
of subsystems are derived from existing vehicles. For 
example, the AMPV uses a common powertrain with the 
Bradley infantry fighting vehicle. 

While the program has released over 90 percent of 
expected design drawings to manufacturing, the overall 
number has fluctuated due to vehicle configuration 
changes. Following its critical design review (CDR) in 
June 2016, the program’s number of expected drawings 
increased by nearly 19 percent. In our 2018 
assessment, we reported that this increase was due to 
the contractor underestimating the number and 
complexity of design drawings needed. For our current 
assessment, however, we found that the program has 
now lowered its total number of design drawings by 19 
percent. According to program officials, this decrease 
reflected a changed design due to testing results. This 
lowered number re-establishes that the program 
released over 90 percent of its design drawings to 
manufacturing by CDR. We have updated our 
assessment of product knowledge table to reflect this 
latest calculation. 

To maintain schedule, the program office delayed some 
of its developmental testing, which it now expects to 
complete in July 2019—over 5 months after it entered 
low-rate initial production and contrary to best practices. 
Remaining technical risk areas include the automatic 
fire extinguishing system and hard transmission shifts, 
both of which may require redesigns if testing reveals 
further deficiencies. 

Production Readiness 

The AMPV program entered low-rate initial production 
in January 2019. The program’s manufacturing 
readiness level did not indicate that its production 
processes were in statistical control, as recommended 
by best practices. Further, the program office did not 
demonstrate its critical manufacturing processes on a 
pilot production line prior to production start. The 
program attributed this to the contractor’s need to 
address manufacturing process deficiencies discovered 
during production of prototype AMPV units. 

The contractor, BAE Systems, delivered all 29 prototype 
vehicles by March 2018—5 months later than earlier 
program plans, which called for delivery of all vehicles 
by October 2017. As a result, the contractor had less 
time than initially planned between the prototype 
deliveries and the start of low rate initial production to 
resolve production issues. Problems with parts 


shortages and changes to engineering drawings, 
among other things, caused the late deliveries. In light 
of these challenges, BAE Systems is modifying its 
overall manufacturing plan to include changes to 
assembly and facility processes. For example, it is 
implementing robotic welding to improve quality and 
manufacturing throughput. In a program assessment 
report, the Defense Contract Management Agency 
(DCMA) noted that although these manufacturing 
modifications should improve the overall process, some 
changes will, in fact, introduce new risks. 

Our review of program documentation shows that the 
program’s ability to meet increased production 
quantities remains a key risk area. In September 2017, 
the Army increased the program’s low-rate initial 
production quantity from 289 to 551 vehicles spread 
over 4 years to support the European Deterrence 
Initiative (EDI)—an effort intended to reassure allies and 
partners of U.S. commitment to their security. According 
to program officials, this quantity has been lowered to 
469 vehicles spread over 3 years with the final EDI 
vehicles being procured as part of full rate production. 
Persisting AMPV technical challenges, coupled with 
contractor plans to produce AMPV in the same facility 
as two other major platforms—the M109A7 Family of 
Vehicles, and M88A2 HERCULES recovery vehicles— 
will together constrain AMPV deliveries. 

Other Program Issues 

Based on recent results from ongoing reliability tests, 
the program reported that it is slightly below its reliability 
goal for the start of low-rate initial production. Program 
officials stated the contractor has addressed AMPV- 
unique issues, but legacy powertrain problems remain. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. Program officials stated that AMPV 
began production ahead of schedule in January 2019 
and is executing activities to show manufacturing 
processes are nearing maturity. According to the 
program, production processes will reach full maturity to 
support full-rate production in fiscal year 2022. To date, 
the program reports that it has implemented production 
process changes that improve vehicle quality and 
provide capacity for increased production quantities. In 
addition, program officials stated that BAE Systems 
produced vehicle hulls ahead of the first low-rate 
production vehicle to validate these process changes. 

At the same time, program officials stated that they 
continue to improve the AMPV design based on recent 
test results. They also stated that vehicle developmental 
tests will finish in July 2019, and they assess the risk of 
additional re-design of the vehicle to be low. Further, 
program officials said that AMPV is meeting its overall 
availability requirement, but reliability is a watch item. 
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Lead Component: Army 


Common Name: CH-47F Block II 



Source: Copyright © 2017 B 


. All rights reserved. 


CH-47F Modernized Cargo Helicopter (CH-47F Block II) 

The Army’s CH-47F Block II program upgrades existing CH-47F 
aircraft intended to provide additional capability, greater reach, and 
increased payload capacity. Improvements include a strengthened 
airframe and drive train, improved flight controls, upgraded fuel and 
electrical systems, and advanced rotor blades that the Army expects 
will increase lift in hot weather conditions. The Army also expects the 
improved fuel and rotor components to reduce operating and support 
costs. CH-47F helicopters provide the Army’s only heavy lift capability 
and are scheduled to remain in service through 2060. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Army 
Program office: Redstone Arsenal, AL 


First full estimate 
(02/2018) 

Latest 

(06/2018) 

Percentage 

change 

Prime contractor: Boeing 

Contract type: CPIF (development) 

Development 

$791.70 

$761.60 

-3.8% 

Software development approach: 

Procurement 

$15,714.80 

$15,768.20 

+0.3% 

Mixed 

Unit cost 

$30.92 

$30.97 

+0.2% 

Next major milestone: Production start 
(August 2021) 

Acquisition cycle 
time (months) 

88 

88 

0.00% 


Total quantities 

542 

542 

0.00% 


Total quantities comprise 3 development quantities and 539 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


S435.82 


Development 



Procurement 



Procurement Quantities 



■ Funded to date 

■ To complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


|Knowledge attained, Q Knowledge not attained, ■information not available, J Not applicable 
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Lead Component: Army 

CH-47F Block II Program 

Technology Maturity 

CH-47F Block II critical technologies continue to 
approach maturity at the same time that the program 
reports that it has released 99 percent of planned 
drawings, which constitutes a stable design. As the 
program continues to mature each technology into a 
final form, fit, and function, the program may need to 
revise its system drawings to accommodate necessary 
changes, which could compromise the program’s 
design stability. 

The program office has identified two critical 
technologies—Ferrium C61 steel and the advanced 
Chinook rotor blade (ACRB)—that it assesses as 
approaching maturity. Ferrium C61 is a new, high 
strength steel designed to create an improved drive 
train (shafts) that can operate under increased 
horsepower, without needing to redesign the CH-47F 
transmission. According to the program office, the CH- 
47F Block II will be the first to use this material in this 
type of application. Other applications include power 
transmission shafts and gears in the aerospace and 
energy industries, as well as off-road vehicles. Program 
officials expect the production cost of Ferrium C61 
shafts to be higher than the steel used in existing CH- 
47F aircraft, but they stated that they will not be able to 
estimate the cost until production processes are 
established. 

The ACRB is a carbon fiber blade intended to improve 
the aircraft’s lift performance while also minimizing 
effects on forward flight. The Block II program is 
tracking technical risks with ACRB. Program officials 
stated they have demonstrated the ACRB in an 
operational environment near or at planned operational 
configuration. Nonetheless, the program characterizes 
the ACRB design as more complex than legacy blades 
due to a new tip design, which has introduced 
manufacturing risk. According to officials, prototypes 
and process changes will help limit that risk. 

Design Stability 

CH-47F Block II released 90 percent of its design 
drawings by critical design review (CDR) in December 
2017. Prior to the CDR, however, the program did not 
elect to developmentally test a fully configured, 
production representative prototype in its intended 
environment. Instead, the program plans to initiate this 
prototype testing in August 2019 (20 months after 
CDR)—an approach inconsistent with best practices. 
Until the program completes this testing, it cannot know 
whether its design is actually stable. 

Production Readiness 

The program plans to enter production in August 2021 
following delivery of three aircraft that officials report 
were procured as part of system development. The 


Common Name: CH-47F Block II 

prime contractor, Boeing, is producing the three system 
development aircraft in the same final assembly facility 
as the current Block I aircraft using many of the same 
processes, tools, and management procedures. The 
program plans to demonstrate statistical control of 
critical manufacturing processes as part of a production 
readiness assessment before starting production, which 
aligns with GAO best practices. Program officials noted 
that Boeing—as the original designer and prime 
manufacturer of the CH-47 series aircraft since 1962— 
has demonstrated statistical control of manufacturing 
processes in previous models. 

The program office further stated that Boeing is the only 
capable source for Block II production. According to the 
program office, Boeing’s significant cargo helicopter 
experience, possession of CH-47F-unique tooling and 
test equipment, and existing Block I production line 
preclude use of full and open competition for Block II 
production. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that the 
CH-47F Block II program is ahead of schedule and 
within expected cost. Specifically, the program office 
stated that following the development contract award, it 
has consistently reported favorable cost and schedule 
performance and met program milestones ahead of 
schedule, which the program office considers indicative 
of overall positive program health. The program office 
also stated that testing of the first system-level 
integrated prototype could begin in July 2020. It also 
said that it conducted two test flights that confirmed 
ACRB design for the development phase of the CH-47F 
program. Further, the program office stated that the 
similarities between the legacy CH-47F and the new 
Block II design allows for demonstration of critical 
technologies before prototype testing. According to the 
program office, this constituted a low-risk approach to 
reduce program cost and schedule. 
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Lead Component: Army 


Common Name: CIRCM 



Source: Northrop Grumman Systems Corporation. 


Common Infrared Countermeasure (CIRCM) 

The Army's CIRCM is the next generation lightweight, laser-based 
infrared countermeasure system for rotary-wing, tilt-rotor, and small 
fixed-wing aircraft across DOD. CIRCM consists of three major items— 
a system processor unit, a pointer tracker, and an infrared laser. 

CIRCM receives input from the Army’s Common Missile Warning 
System and employs the pointer tracker to track incoming missiles. It 
jams the missile by using laser energy, thus causing the missile to miss 
the aircraft. CIRCM is to replace the Advanced Threat Infrared 
Countermeasures system and enable integration of laser-based 
infrared countermeasures onto smaller aircraft. 
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for public release) 


Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Army 
Program office: Huntsville, AL 


First full estimate 
(07/2016) 

Latest 

(11/2018) 

Percentage 

change 

Prime contractor: Northrop Grumman 
Contract type: CPFF/FPI/FFP 

Development 

$802.50 

$880.60 

+9.7% 

(development and low-rate initial 

Procurement 

$1,895.70 

$3,094.50 

+63.2% 

production) 

Unit cost 

$2.40 

$2.18 

-9.2% 

Software development approach: 

Incremental 

Acquisition cycle 

115 

117 

+1.7% 

Next major milestone: Start operational 
testing (June 2019) 

time (months) 

Total quantities 

1,124 

1,829 

+62.7% 


Quantities comprise 48 development quantities and 1,781 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$254.74 ^ $625.87 


0 


Procurement 


$ 3 . 088 . 26 ^^ ^^ $6 ~ 


Procurement Quantities 


1781 


O 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
|To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 



Page 79 


GAO-19-336SP Weapon Systems Annual Assessment 











































Common Name: CIRCM 


Lead Component: Army 

CIRCM Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The CIRCM program has fully matured its six critical 
technologies and stabilized its design. However, 4 
months into production, the program had yet to 
demonstrate that its critical manufacturing processes 
were in statistical control—an approach inconsistent 
with best practices. 

Following development start, the program demonstrated 
its six critical technologies by integrating a prototype 
CIRCM system on an Army aircraft and testing it in a 
realistic environment. The program has also released 
100 percent of design drawings, which indicates a 
stable design. Nonetheless, persistent reliability 
shortfalls with CIRCM units—and the design changes 
potentially required to remedy these shortfalls—pose 
risk to CIRCM’s design stability. 

In April 2017, the DOD Inspector General found 
deficiencies with the CIRCM program office’s plan for 
demonstrating minimum reliability requirements. The 
Inspector General recommended that the Army revisit 
CIRCM performance requirements and test plans prior 
to production start. In response, the Army took actions 
that included updating system reliability requirements in 
the CIRCM’s master test plan. This update set a new 
goal for the end of the system’s initial operational test 
and evaluation phase in November 2019—achieving150 
hours “mean time between operational mission failure” 
with an 80 percent likelihood of operating for 150 hours 
without a failure. Program officials stated that meeting 
this goal will position the CIRCM to demonstrate 214 
hours mean time between operational mission failure as 
required for the program’s June 2020 full-rate 
production decision. 

At the time of its low-rate initial production start in 
September 2018, the CIRCM demonstrated 148.5 hours 
reliability at a 50 percent likelihood of performing 
reliably for 148.5 hours without failure. More recently, 
the program reported that it has increased CIRCM’s 
reliability performance to 151 hours within the same 50 
percent likelihood. In a September 2018 briefing it 
prepared for the CIRCM production decision, the Army’s 
Aircraft Survivability Equipment project management 
office that oversees the CIRCM program office stated 
that, should CIRCM ultimately fall short of the 214-hour 
requirement, the consequences will likely be delayed 
entry into full-rate production and postponement in 
equipping the first Army unit with CIRCM. 

The program office reported that, at the time of 
CIRCM’s entry into low-rate initial production, the 
contractor, Northrop Grumman, had achieved 
manufacturing readiness at the level recommended by 
DOD guidance. That guidance calls for programs to 
demonstrate critical manufacturing processes on a pilot 
production line, but does not require statistical control of 


those processes until the full-rate production decision. 
Our prior work has found that this DOD standard falls 
short of the industry best practice. 

Further, a June 2018 production readiness review 
identified concerns with (1) Northrop Grumman’s ability 
to oversee subcontractors and (2) the pointer tracker 
subcontactor’s ability to deliver reliable components on 
schedule. The Office of the Secretary of Defense 
expressed similar concerns and found that the pointer 
tracker subcontractor may be challenged to increase 
production from 6 units per month to the 26 per month 
needed to support low-rate production and testing. The 
Army also is tracking risks that Northrop Grumman 
cannot provide timely delivery of CIRCM assets for 
testing and that production facilities for the pointer 
tracker component do not have the capacity to meet 
demand for full-rate production. The program has 
mitigation efforts for both risks under way, which 
include, among other things, increasing repair capability 
at Northrop Grumman and adding to the workforce at 
the pointer tracker production facility. It expects to retire 
the testing assets risk in May 2019 and the pointer 
tracker production risk in October 2019. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
stated that CIRCM completed all engineering and 
manufacturing development activities and entered 
production in September 2018. According to the 
program office, CIRCM exceeded all key performance 
requirements as demonstrated through rigorous 
laboratory, aircraft flight, and live fire missile testing. 

The program office also stated that the Army Test and 
Evaluation Command Director assessed CIRCM to be 
an effective system and on track to be suitable and 
survivable. Further, the office stated that the program 
has awarded a low-rate initial production contract and 
identified plans to begin initial operational test and 
evaluation in June 2019. According to the program 
office, it continues to aggressively manage contractor 
performance to facilitate a successful transition to full- 
rate production. Additionally, the Army stated that the 
CIRCM program is on track to achieve the 214-hour 
requirement necessary for full-rate production, and that 
it will reassess program strategy going forward if 
CIRCM does not achieve this goal. 
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Lead Component: Army 


Common Name: HMS 



Source: U.S. Army. 


Handheld, Manpack, and Small Form Fit Radios (HMS) 

Through its HMS program, the Army is procuring software-defined 
radios that will connect with existing radios and increase the service’s 
communications and networking capabilities. The program continues 
efforts begun under the former Joint Tactical Radio System program to 
procure multiple radios, including the Handheld (Leader and Rifleman) 
and the Manpack. A subset of Manpack radios will operate with the 
Mobile User Objective System (MUOS)—a worldwide, multiservice 
Navy satellite communication system. In 2017, the Army deferred its 
acquisition of one-channel Rifleman radios in favor of two-channel 
Leader radios. 
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Program Essentials 


Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Army 
Program office: Aberdeen Proving 
Ground, MD 

Prime contractors: General Dynamics 
C4 Systems, Inc.; Harris Corporation; 
Collins Aerospace; Thales Defense and 
Security 

Contract types: CPAF (development); 
FFP/IDIQ (low-rate initial and full-rate 
production) 

Software development approach: Other 
(modifications to existing systems) 

Next major milestone: Full-rate 
production (February 2021) 



First full estimate 
(05/2004) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$609.90 

$1,421.00 

+ 133.0% 

Procurement 

$10,629.60 

$7,977.20 

-25.0% 

Unit cost 

$0.03 

$0.04 

+2.4% 

Acquisition cycle 
time (months) 

85 

124 

+45.88% 

Total quantities 

328,674 

271,202 

-17.5% 

Acquisition cycle time reflects development start to initial operational capability for the Manpack radio (August 2014). 
Total quantities comprise 833 development quantities and 270,369 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 


$6,360.80 



Procurement Quantities 



■ Funded to date 

■ To complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ■information not available, J Not applicable 
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Common Name: HMS 


Lead Component: Army 

HMS Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The HMS program office is now acquiring the Leader 
and Manpack radios as non-developmental items, 
which has precluded the program’s use or tracking of 
any critical technologies. When the HMS program 
started development in 2004, it did not assess the 
maturity of the technologies it then deemed critical—an 
approach inconsistent with best practices. The program 
had also not fully matured its critical technologies by its 
2011 production start. Instead, the program completed 
development of individual critical technologies in May 
2015. This divergence from best practices likely 
contributed to the over 130 percent increase in 
development costs since the first full estimate. 

At its 2008 critical design review, the program office had 
completed fewer than half of its planned design 
drawings, which did not meet best practices criteria for 
design stability. Most importantly, the program’s 
persistent technology immaturity between 2004 and 
2011, including at the critical design review, contributed 
to radio designs that did not fully accommodate the final 
form, fit, and function of critical technologies as they 
matured. These design shortfalls became evident as 
HMS radios entered testing, with testers identifying 
significant reliability and suitability problems with radios 
at that time. 

With the move to a non-developmental acquisition— 
including their decision to shift to a two-channel radio 
and introduce an additional waveform—the program 
has developed a new test plan. The Army has started 
working with the contractors to perform customer and 
qualification testing to ensure their radios meet Army 
specifications. These tests will support preparations for 
operational testing, which is scheduled to begin in June 
2020. 

The HMS program has not yet finalized its assessments 
of contractor manufacturing readiness levels under the 
restructured program. To date, the HMS program office 
completed preliminary assessments of the Harris 
Corporation’s and Collins Aerospace’s readiness to 
begin production for the Manpack radio and found them 
to be on track to support production decisions. Further, 
program officials stated that they already held a follow¬ 
up assessment of Harris Corporation and plan to hold 
one with Collins Aerospace in June 2019. Officials 
reported that the program added Leader radios to the 
existing program in summer 2018 and that they plan to 
assess production readiness of Leader in late fiscal 
year 2019. 

Other Program Issues 

At present, warfighters are not able to use the MUOS 
waveform—which some Manpack radios will rely on— 
because the MUOS program is still addressing 


deficiencies found in initial operational testing in 
November 2015. Over the past several years, problems 
with MUOS testing have repeatedly delayed availability 
of the waveform. Although the program office has not 
identified MUOS as a critical technology, without this 
waveform, affected Manpack radios will be able to 
communicate only through legacy communications, 
which limit the capacity of the network. The Navy 
currently plans additional MUOS testing for mid-2019, 
but further delays to the waveform could lead to delays 
in fielding affected Manpack radios by March 2020, 
which is already delayed from initial plans by nearly 2 
years. Program officials told us Manpack radio low-rate 
initial production will move forward regardless of the 
MUOS waveform performance during the Navy's testing 
scheduled in 2019. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. According to the program office, in 
September 2018, it added the two-channel Leader 
Radio capabilities to the existing Handheld indefinite- 
delivery/indefinite-quantity contracts with two vendors, 
Harris Corporation and Thales Defense & Security. The 
program office also said that, based on the Army 
Network Review, it amended the Leader Radio 
capability production document to address threshold 
radio waveforms, weight, and size. Additionally, it stated 
that it is also amending the Manpack capability 
production document to address waveform 
clarifications, and that this documentation is currently 
pending approval from Army leadership. According to 
the program office, these amended documents will 
validate and stabilize requirements consistent with 
current non-developmental item capabilities and the 
HMS acquisition strategy. The program also said that it 
awarded low-rate initial production delivery orders to 
procure 3,800 Leader radios and an additional low-rate 
delivery order to procure 2,258 Generation 2 Manpack 
radios. 
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Lead Component: Army 


Common Name: IAMD 



Source: Northrop Grumman. 


Integrated Air and Missile Defense (IAMD) 

The Army's IAMD program plans to network sensors, weapons, and a 
common battle command system across an integrated fire control 
network. Its purpose is to support the engagement of air and missile 
threats. The IAMD battle command system will provide a capability for 
the Army to control and manage IAMD sensors and weapons, such as 
the Sentinel radar and Patriot launcher and radar, through an interface 
module that supplies battle management data and enables networked 
operations. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Under 
Secretary of Defense for Acquisition and 
Sustainment 

Program office: Redstone Arsenal, AL 


First full estimate 
(12/2009) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$1,786.00 

$3,314.20 

+85.6% 

Prime contractor: Northrop Grumman 
Space & Mission Systems Corporation 

Procurement 

$3,844.20 

$3,872.70 

+0.7% 

and Raytheon 

Unit cost 

$19.02 

$15.14 

-20.4% 

Contract type: CPIF/CPFF/FPI 
(development) 

Acquisition cycle 
time (months) 

80 

148 

+85.0% 

Software development approach: 

Mixed 

Next major milestone: Low-rate initial 
production (September 2020) 

Total quantities 

296 

479 

61.8% 

Total quantities comprise 25 development quantities and 454 procurement quantities. 



Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$676.91 ^ $2,637.26 




Procurement 

$3,851.37 _ $21.34 


o 


Procurement Quantities 


454 


O 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
|To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


|Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Lead Component: Army 

IAMD Program 

Technology Maturity and Design Stability 

The IAMD program has demonstrated that its four 
critical technologies are mature and that its system 
design is stable. Since our 2018 assessment, the 
program changed two IAMD command and control 
components that program officials said were reaching 
end-of-life and becoming unreliable. This change 
contributed to a 29 percent reduction in the program’s 
total number of expected design drawings. To date, the 
program has completed 96 percent of its revised total 
number of expected design drawings, which constitutes 
a stable design. 

IAMD battle command system (IBCS) software 
performance indicated the need for several 
improvements including more soldier training. Program 
officials stated that the first exercises to address this 
and involve warfighters in August 2017 and October 
2017 were successful and that the results of 2018 tests 
also demonstrated progress over previous tests. At the 
same time, program officials acknowledged that 
upcoming tests will require additional refinements. 
Despite these mixed results, the program office is 
confident that software reliability is on track in part 
because the number of software failures the program 
has experienced has decreased and software deliveries 
are on schedule. However, our review of analysis of the 
test results indicated that the software continues to 
need several improvements, including to display and 
tracking capabilities.These and additional 
developmental tests are in preparation for a second 
limited user test (referred to as limited testing) in early 
2020. The program is conducting a second test 
because it demonstrated an unsatisfactory performance 
of the IBCS software in 2016 during the initial limited 
testing. 

The program office plans to begin IBCS qualification 
testing in mid-2019, followed by the start of the next 
major software build. To help further reduce the risk of 
IBCS deficiencies during the 2020 test, program officials 
stated the contractor is testing IBCS software with 
tactical network and weapon/sensor interfaces prior to 
government acceptance so that issues can be 
addressed before acceptance. 

Production Readiness 

The program plans to conduct a manufacturing 
readiness assessment in fiscal year 2020, in 
preparation for the September 2020 low-rate initial 
production decision. However, the program has allotted 
only a few months between the planned completion of 
the limited test in mid-2020 and the September 2020 
production decision. Officials stated that the program 
recently began piloting an Agile software development 
approach, which provides opportunities for the program 
office to correct software deficiencies that may arise 


Common Name: IAMD 

during the limited test in a shorter timeframe. This 
strategy does not fully account, however, for any 
hardware deficiencies, or for more severe software 
deficiencies, that may be discovered during the test. 
Should either of those occur, the program will likely face 
a choice: postpone its low-rate initial production 
decision to afford time to resolve deficiencies or enter 
low-rate initial production with system performance that, 
at present, it would deem unsatisfactory. 

Other Program Issues 

After experiencing cost growth in excess of the limits 
authorized in its acquisition program baseline, in 
December 2017 the Army received approval from the 
Under Secretary of Acquisition, Technology, and 
Logistics to restructure the program. The Army is now 
executing the program within revised cost and schedule 
parameters and, in accordance with the Under 
Secretary’s direction, plans to update its acquisition 
program baseline at the time of the low-rate initial 
production decision in September 2020. At that point, 
program officials said, a new independent cost estimate 
will also be available. 

The program also noted that hardware delays for the 
government software integration lab may threaten the 
schedule leading up to its low-rate initial production 
decision. According to program officials, the program 
has mitigated hardware shortages by implementing 
incremental deliveries of hardware. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. Program officials stated that the 
program has received hardware and software on 
schedule to support training and testing, and is 
continuing to develop key performance capabilities 
needed to support the low-rate production decision. 
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Lead Component: Army 


Common Name: ITEP 



Source: U S. Army. 


Improved Turbine Engine Program (ITEP) 

The Army’s ITEP is developing a replacement engine for the Black 
Hawk and Apache helicopter fleets. The new engine is being designed 
to provide increased power, performance, and fuel efficiency; 
enhanced reliability; increased service life; and a lower maintenance 
burden. The Army plans to field the new engine in fiscal year 2027. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Army 
Program office: Redstone Arsenal, AL 


First full estimate 
(N/A) 

Latest 

(10/2018) 

Percentage 

change 

Prime contractor: Advanced Turbine 
Engine Company; General Electric 

Development 

N/A 

$1,913.30 

N/A 

Company 

Procurement 

N/A 

$8,322.50 

N/A 

Contract type: FPI (design) 

Unit cost 

N/A 

$1.73 

N/A 

Software development approach: Agile 
development 

Acquisition cycle 

N/A 

103 

N/A 

Next major milestone: Design review 
(March 2020) 

time (months) 

Total quantities 

N/A 

5,994 

N/A 


ITEP received approval to enter development in January 2019, but has yet to approve a program baseline, which 
would identify a first full estimate. Total quantities comprise 69 development and 5,925 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$1.435.15 ^ $478.14 


u 


Procurement 
$8.322.49 


o 


Procurement Quantities 


5,925 


O 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


Funded to date 
|To complete 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Lead Component: Army 

ITEP Program 

Technology Maturity 

The ITEP program office received approval to enter 
system development in January 2019 with five critical 
technologies approaching full maturity. Because ITEP 
had not established a baseline as of the end of January 
2019, the program’s attainment of product knowledge 
could not be assessed at development start. ITEP 
entered technology development in August 2016 with 
the award of two contracts to General Electric and the 
Advanced Turbine Engine Company. During this phase, 
the program office identified three critical technologies: 
the advanced inlet particle separator, compressor 
advanced aerodynamics, and hybrid bearings. Both 
contractors incorporated these technologies into their 
prototypes for full-engine testing. A technology 
readiness assessment by the Office of the Assistant 
Secretary of the Army (Acquistion, Logistics, and 
Technology) (ASA (ALT)) in June 2018 concluded that 
these three technologies were nearing full maturity. 

After completing an independent review of preliminary 
designs, the Office of the Chief Systems Engineer 
within ASA (ALT) identified two additional engine 
components as critical technologies, based on the 
contractors’ use of additive manufacturing. Additive 
manufacturing relies on 3D printing, and is often used in 
rapid prototyping and direct digital manufacturing. 
Because both contractors are using additive 
manufacturing in other programs, the review team 
concluded that these two technologies are also 
approaching full maturity. The Army plans to further 
mature all five critical technologies through engine 
testing beginning in fiscal year 2021,2 years after the 
start of system development. 

Design Stability 

In spring 2018, the ITEP program office conducted 
individual preliminary design reviews with both 
contractors to assess each design. In August 2018, an 
independent review team from the Office of the Chief 
Systems Engineer, ASA (ALT), also assessed each 
design. The independent review concluded that 
preliminary design reviews lacked objective and 
quantifiable entrance criteria. In addition, it found that 
the design reviews of lower level elements could not 
fully inform the system-level review because the 
schedule was compressed. Despite these findings, the 


Common Name: ITEP 

independent review team concluded that the program 
office’s efforts to close deficiencies identified in the 
preliminary design reviews were sufficient and 
concluded the program was ready to proceed into 
system development. Following the start of system 
development, the program office plans to select one 
vendor and hold a critical design review in March 2020. 

Production Readiness 

An independent technical review conducted by the 
Under Secretary of Defense for Research and 
Engineering in October 2018 assessed manufacturing 
risk as low. This review concluded that the program 
demonstrated all manufacturing processes—to include 
additive manufacturing—in a production-relevant 
environment, on the basis that each contractor has 
used these processes in conventional manufacturing. 
The program currently plans to begin production in July 
2024. 

Other Program Issues 

ITEP is highly dependent on concurrent development 
efforts by the engine vendors and the aircraft 
manufacturers, and this dependency has limited early 
design analysis. During technology development, both 
engine vendors conducted trade studies with the aircraft 
manufacturers to develop a proposed interface control 
document for their respective designs. However, the 
vendors could not fully assess how their designs 
conformed with the structural and environmental 
constraints of the aircraft.The program plans to have the 
proposed interface control document finalized during 
system development when the selected engine vendor 
and the aircraft manufacturer conducts a detailed 
structural and environmental analysis to support 
integration. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
stated that it awarded the engineering and 
manufacturing development contract to General Electric 
on February 1, 2019. According the program office, it is 
providing incentives to the contractor to accelerate 
ITEP’s schedule and thus allow integration into the 
Black Hawk and Apache helicopters and other aircraft. 
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Lead Component: Army 


Common Name: JAGM 



Source: U S. Army. 


Joint Air-to-Ground Missile (JAGM) 

The Joint Air-to-Ground Missile is an Army-led program with joint 
requirements from the Navy and Marine Corps. The missile is designed 
to be air launched from helicopters and unmanned aircraft systems to 
target tanks, light armored vehicles, missile launchers, bunkers, and 
buildings. It is intended to provide precision attack capabilities no 
matter the time of day or weather conditions. JAGM will replace all 
Hellfire missile variants. 
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Program Essentials 

Milestone decision authority: Army 
Program office: Redstone Arsenal, AL 
Prime contractor: Lockheed Martin 

Contract type: FPI (development and 
low-rate initial production) 

Software development approach: 

Incremental 

Next major milestone: Start operational 
test (March 2019) 


Program Performance (fiscal year 2019 dollars in millions) 


First full estimate 
(09/2015) 

Latest 

(12/2018) 

Percentage 

change 

Development 

$1,040.60 

$1,113.20 

+7.0% 

Procurement 

$4,989.30 

$5,958.00 

+19.4% 

Unit cost 

$0.23 

$0.27 

+18.0% 

Acquisition cycle 

38 

44 

+15.8% 

time (months) 




Total quantities 

26,437 

26,437 

0.00% 


Total quantities comprise 118 development quantities and 26,319 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 


$ 5 . 630.22 



Procurement Quantities 



■ Funded to date 

■ To complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Common Name: JAGM 


Lead Component: Army 

JAGM Program 

Technology Maturity and Design Stability 

The JAGM program has matured its three critical 
technologies and stabilized its system design. The 
program office assessed all technologies—guidance 
seeker assembly/sensor platform, sensor software, and 
mission software—as mature in 2016 following a series 
of successful ground tests and one flight test from a 
MQ-1C Gray Eagle. JAGM components—including the 
motor, warhead, and electronics—have already been 
fully matured for the existing Hellfire missile, thus 
decreasing development and manufacturing risks. 

The program reported that JAGM had a stable design at 
its January 2016 critical design review. Since our 2018 
assessment, however, the program changed the total 
number of JAGM drawings, including adding 10 
drawings related to updated parts, operations, and 
calibration procedures. This is the third year in a row in 
which the program has reported an increase in total 
design drawings. As a result of this drawing growth, we 
found that the program office had released 89 percent 
of JAGM drawings at the critical design review—an 
amount that falls just short of the 90 percent level 
recommended by best practices. We have updated our 
Attainment of Product Knowledge table to reflect this 
change in design stability from our previous 
assessment. The program reported that it has now 
released all planned drawings for JAGM. 

Production Readiness 

The JAGM program entered low-rate initial production in 
June 2018, at which time it had achieved manufacturing 
readiness at the level recommended by DOD guidance. 
That guidance calls for programs to demonstrate critical 
manufacturing processes on a pilot production line, but 
does not require statistical control of those processes 
until the full rate production decision. Our prior work has 
found that this DOD standard falls short of the industry 
best practice. 

The program exercised contract options at the end of 
fiscal year 2018 and beginning of fiscal year 2019 for 
low-rate production totaling 1,423 missiles. A program 
official told us that the contractor began delivering the 
first lot of 373 low-rate production missiles in December 
2018. The program expects 1,050 missiles from the 
second and third lots to be delivered by November 
2021. The program plans to exercise another contract 
option for 796 low-rate production missiles in the fourth 
quarter of fiscal year 2019. Program officials told us that 
due to limited funding, they will purchase fewer missiles 
in the fourth lot than originally planned. 

Since our 2018 assessment, the program office delayed 
its full-rate production decision by 7.5 months to allow 
for the collection and review of additional cost and 
software data. This additional information was needed 
to inform the full-rate production decision. 


Other Program Issues 

In April 2016, the Army delayed initial operational 
testing and evaluation for JAGM by 2 years as the AH- 
64 Apache helicopter's software—used to launch 
Hellfire missiles—required more pilot input than 
expected. The Apache program office has since 
developed and tested new platform software that will 
enable pilots to more easily select the full range of 
options. The program began integrated testing of the 
missile in January 2019. 

The program completed 49 developmental test firings in 
March 2018—including 10 firings during a limited test 
under operational conditions—to support its June 2018 
low-rate production decision. The program also 
identified several JAGM deficiencies during the test. 
According to program officials, all of the deficiencies 
identified have been addressed and successfully 
resolved, including a missile failure. They also stated 
that the program was able to successfully demonstrate 
firing two missiles consecutively in January 2019, and 
that they had determined that a previous missile failure 
was due to the incorrect sequencing of actions in the 
cockpit. 

In December 2018, the program office reported a 
significant cost increase from its prior estimate, which it 
stated was because of changes in program 
requirements, changes in cost estimating 
methodologies, and an extended procurement 
schedule. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program provided 
technical comments, which we incorporated where 
appropriate. The program office stated that the 7.5- 
month delay in the full-rate production decision has not 
negatively affected the program in any way. The 
program office also said that it will use the additional 
cost and software information it is now gathering to 
better inform that full-rate production decision. At the 
same time, the program office reported that it remains 
on track to award a full-rate production contract in 
September 2020, as previously planned. 
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Lead Component: Army 


Common Name: JLTV 



Source: U.S. Army. 


Joint Light Tactical Vehicle (JLTV) 

The Army and Marine Corps’ JLTV is a family of vehicles developed to 
replace the High Mobility Multipurpose Wheeled Vehicle (HMMWV) for 
some missions. The JLTV is expected to provide protection for 
passengers against current and future battlefield threats, increased 
payload capacity, and improved automotive performance over the up- 
armored HMMWV variant, which includes an armor package. It is 
designed to be transported by air or ship. Two- and four-seat variants 
are planned with multiple mission configurations. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Army 
Program office: Harrison Township, Ml 


First full estimate 
(10/2012) 

Latest 

(06/2018) 

Percentage 

change 

Prime contractor: Oshkosh Defense, 

LLC 

Development 

$1,065.90 

$988.50 

-7.3% 

Contract type: FFP (production) 

Procurement 

$24,127.20 

$22,839.20 

-5.3% 

Software development approach: 

Unit cost 

$0.46 

$0.41 

-10.5% 

Waterfall 

Next major milestone: Full-rate 

Acquisition cycle 

125 

144 

+15.2% 

production (May 2019) 

time (months) 

Total quantities 

54,730 

58,306 

+6.5% 


Total quantities comprise 116 development quantities and 58,190 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$77.31 ^ _ $911.20 


o 


Procurement 

$20.596.54 ^ $2.242.66 


o 


Procurement Quantities 
5,620 


52.570 o 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
| To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Lead Component: Army 

JLTV Program 

Technology Maturity and Design Stability 

The JLTV program has matured its two critical 
technologies and stabilized the system design. 

However, the program’s discovery of significant 
deficiencies during operational testing—and the 
corrections that those now require—pose risk to 
maintaining that design stability. 

Following our 2018 assessment, the program’s total 
number of drawings increased to accommodate needed 
retrofits, capability changes, and delayed release of 
vehicle parts that support JLTV fielding. These drawing 
increases caused the program to fall just short of the 
best practice of 90 percent design drawings released at 
critical design review. We have updated our Attainment 
of Product Knowledge table to reflect this change. 

Production Readiness 

Program officials conducted a manufacturing readiness 
assessment in September 2018 that found that the 
prime contractor, Oshkosh Defense, had matured 
production processes and demonstrated high 
manufacturing readiness levels. Program officials stated 
that, during low-rate initial production, Oshkosh 
significantly reduced the number of defects per 
manufactured vehicle, from 14.6 in September 2016 to 
1.3 in September 2018. Program officials also stated 
that Oshkosh has provided on-time deliveries for 6 
consecutive months and is now producing vehicles 2 
months ahead of schedule at a rate of about 11 per day. 
The program is also utilizing statistical process controls 
to demonstrate ongoing JLTV production readiness. 

In October 2018 program officials had yet to accept 39 
vehicles because of deficiencies related to their 
Warfighter Information Network-Tactical (WIN-T) kit 
configuration, which provides satellite communication 
capabilties to the user. Program officials stated they will 
not accept these vehicles until Oshkosh and the WIN-T 
program resolve the deficiencies. 

Other Program Issues 

The Army and Marine Corps recently concluded 
operational testing for JLTV and found the vehicles to 
be survivable for the crew and effective for small 
combat and transport missions, but not operationally 
suitable because of their high maintenance needs, low 
reliability, training and manual deficiencies, and safety 
shortcomings. The Director, Operational Test and 
Evaluation, made the same findings as the Army and 
Marine Corps. Army operational testers recommended 
a conditional release to full-rate production for most of 
the vehicle variants, pending resolution of all suitability- 
related deficiencies. Marine Corps operational testers, 
however, found the JLTV could support their mission 
and advocated for the program’s unconditional 
transition into full-rate production. However, the Army 


Common Name: JLTV 

delayed the full-rate decision from December 2018 to 
May 2019 to review new potential requirements. 
According to officials, the review will not affect fielding 
and existing vehicles will be retrofitted if the 
requirements are incorporated. Officials also said they 
addressed some of the deficiencies discovered during 
testing by extending training, adjusting vehicle 
maintenance tasks, authenticating manuals, and 
correcting door and latch safety issues. 

Program officials previously reported to us that the 
JLTV engineering drawings were inadequate to facilitate 
open competition for production in fiscal year 2021 and 
beyond. Following our 2018 assessment, program 
officials stated they were concerned about the quality of 
the engineering drawings and therefore changed 
management and engineering processes in order to 
give Oshkosh improved feedback for fixing the 
drawings. As a result, program officials stated that they 
have seen significant improvements to the drawings. 
Program officials stated they have validated 90 percent 
of the drawing modifications and anticipate completing 
validation by March 2019. Program officials stated the 
drawings are now primarily government-validated and 
can be used in future competitions for contracts to build 
or support vehicles to government specifications. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. 
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Lead Component: Army 


Common Name: IFPC Inc 2-1 Block 1 



Source. U.S. Army. 


Indirect Fire Protection Capability Increment 2-Intercept 
Block 1 (IFPC Inc 2-1 Block 1) 

The Army’s IFPC Inc 2-1 is a follow-on effort to enhance and extend 
the range of the first IFPC increment, which provided a short-range 
capability to counter threats from rockets, artillery, and mortars. IFPC 
Inc 2-1 consists of four separate subsystems—an existing sensor, 
interceptor (missile), and fire control system, and a new multi-mission 
launcher the Army plans to develop—in three blocks. We assessed 
Block 1, which the Army expects will counter cruise missiles and 
unmanned aircraft. 
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Program Essentials 

Milestone decision authority: Army 
Program office: Huntsville, AL 
Prime contractor: Rafael Advanced 
Defense Systems 
Contract type (planned): FFP 
(procurement of interim system) 
Software development approach: NA 

Next major milestone: Procurement 
contract award for interim system (May 
2019 ) 


Estimated Program Cost 

(FY 2019 dollars in millions) 



Current Status 

The Army is pursuing a two-phased approach to acquiring IFPC Inc 2-1 
Block 1 capabilities. According to program officials, the Army identified two 
critical problems in February 2018 as the program prepared to enter system 
development. Specifically, Raytheon—the developer of the original planned 
interceptor—determined that the interceptor would shut down during 
operations to avoid overheating, resulting in a failure to launch. In addition, 
Army warfighters expressed concerns regarding the difficult procedures 
needed to reload the interceptor into the launcher. As a result of these 
discoveries, the Army’s Air and Missile Defense cross-functional team 
reviewed the program and recommended alternative approaches. 

In addition, in response to legislation the Army certified that there is a need 
for an interim missile defense capability and plans to deploy two batteries 
capable of fixed-site cruise missile defense—the required capability of IFPC 
Inc 2-1—by September 30, 2020. In the short term, to meet this schedule 
and provide interim capabilities, the Army plans to acquire two Israeli Iron 
Dome batteries from Rafael. Program officials stated that they are planning 
interoperability experimentation for these batteries in fiscal year 2019 with 
the Sentinel radar and Integrated Battle Command System. Depending on 
the outcome of these experiments, the Army may elect to develop a new, 
interoperable launcher for the long term, which it would plan to field by 
2023. 


■ Development ■ Procurement 


Planned Quantities 



■ Development ■ Procurement 

Note: Estimated program costs reflect fiscal year 
2019 and 2020 amounts only. Planned quantities 
reflect Iron Dome quantities only. 


Attainment of Technology Maturation Knowledge 

As of January 2019 


Conduct competitive prototyping 

O 

O 

Complete technology readiness assessment 

O 

Validate requirements 

Complete preliminary design review 

O 


Not applicable 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated where appropriate. 
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Source: U S. Army 


Precision Strike Missile (PrSM) 

The Army’s Precision Strike Missile (PrSM) (formerly known as Long 
Range Precision Fires) will be a ballistic missile designed to attack 
area and point targets at planned ranges of 400 kilometers or more. 
The Army anticipates that each PrSM missile container will hold two 
missiles for launch. The Army plans to design PrSM, as one of a 
family of munitions, to be compatible with existing M142 and M270A2 
rocket launcher systems and to comply with statutory requirements for 
insensitive munitions and DOD policy on cluster munitions. 
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Program Essentials 


Current Status 


Milestone decision authority: Army 
Program office: Redstone Arsenal, AL 
Prime contractors: Lockheed Martin; 
Raytheon 

Contract type: Other transaction 
agreement (technology maturation and 
risk reduction) 

Software development approach: Agile 
development 

Next major milestone: Development 
start (June 2021) 


Estimated Program Cost 


The Army worked to maintain PrSM competitive prototyping amidst funding 
constraints. The Army plans to down-select from two prime contractor 
teams and prototypes to one in 2021 prior to development start. According 
to the program office, it requested and the Army provided funding to 
facilitate these competitive prototyping plans. 

The Army identified PrSM as a priority and has accelerated the missile’s 
acquisition schedule to provide an early capability in fiscal year 2023. 
However, the missile will be usable only from the M142 launcher by then. A 
program official stated that legacy systems on the M270 launcher need to 
be updated before PrSM software can be integrated. 

The Army continues to refine PrSM performance requirements. In 
December 2018, the Army increased the range requirement from 300 to 
400 kilometers and launcher quantities from one to two. 


(FY 2019 dollars in millions 


Attainment of Technology Maturation Knowledge 


$ 2 , 025.7 



As of January 2019 


Conduct competitive prototyping 

• 

Complete technology readiness assessment 

€ 

Validate requirements 

€ 

Complete preliminary design review 

• 


^^nowledg^ttained^^Tiowledg^lanned^^P^nowledg^ioTtained^^^nfomiatiorMio^Tlable^^^T^ppNcabl^ 


■ Development ■ Procurement 


Planned Quantities 


2,422 



56 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated where appropriate. Program officials stated that the Army 
continues to maintain competition and accelerate the program. According to 
the program office, the current timeline reflects acceleration of the 
approved schedule, in accordance with Army leadership’s desire to field a 
capability in fiscal year 2023, complete testing in fiscal year 2024, and 
begin full rate production. The program office said that it continues efforts to 
accelerate PrSM’s integration with the M270 launcher. 


■ Development ■ Procurement 
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NAVY and 
MARINE CORPS 
PROGRAM 
ASSESSMENTS 


GAO Assessed 21 of the 
Navy’s 42 Current Major 
Defense Acquisition 
Programs 


Current program-1 

(Cost baseline not 
established) 


Other programs 
(Not assessed) 



The 21 Current Programs GAO 
Assessed Represent 70 Percent of 
the Navy’s $885 Billion Estimated 
Total Acquisition Costs 

(Fiscal year 2019 dollars in billions) 


Current 

programs 



■ We also assessed two future major defense acquisition programs. We did not assess programs that 
had yet to enter technology development or had already progressed into full-rate production. 


Most Current Programs GAO Assessed 
Have Either Cost Growth, Schedule 
Delays, or Both 


Total Acquisition Cost Performance 
in Current Programs Varied Among 
Knowledge Points (KP) 


Most Current Programs Did Not Attain 
Knowledge at Key Points 


Program 

complete 

jL. 

Programs in 
production 


T-AO 205 

AMDR 
’OASuW 


IRST# 



• LHA8 


• DDG1000 iM 
LCSMM 


P-8A 


MQ-4< 


G/ATOR* 

SSC 


© 


• LPD17 


Programs in system 
development 


(Fiscal year 2019 dollars in billions) 


$ 12.1 





$7.0 

-$0.6 



KP1 to 

KP2 to 

Post - 

KP2 

KP3 

KP3 


KP1 

t „ 


1 of 16 

current 

programs 

attained 

knowledge 


KP2 

* 1 of 15 

WVVVVWWV current 

programs 
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* • W W W W • knowledge 
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| We did not assess knowledge attainment for five current programs that are well into production, or for one current 
program that did not have a baseline as of December 2017. 


• ACV 
•VH-92A 


| For shipbuilding programs, we correlated detail design contract award, fabrication start, and lead ship delivery with 
development start, critical design review, and production start, respectively. Shipbuilding programs are excluded from 
KP3 because our best practices do not identify a corresponding knowledge point for ships. 


NGJ 

Mid 


Band 


SSBN 

826 


JPALS 


Average Acquisition Cycle Time of Current Programs Increased 
by 29 Months from First Full Estimate (FFE) to Current Estimate 


0 5 10 15 20 25 30 35 40 

Program age 

• Cost growth of more than 15 percent and/or 
schedule delays of more than 6 months 

• Cost growth of 15 percent or less and/or 
schedule delays of 6 months or less 

• Cost growth of 0 percent or less and 
schedule delays of 0 months or less 
Note: Bubble size is based on each program’s currently 
estimated future acquisition funding needed 


Programs' 
average FFE 
cycle time 



Number of months 


Programs' average 
cycle time change from 
FFE to current estimate 


| One current program—MQ-25—entered system 
development after December 2017 and therefore 
is excluded from our cost, schedule, knowledge 
attainment, and contract analysis. 


Cost and schedule analysis is primarily based on estimates from December 2017 selected acquisition reports. 
This information may differ from the amounts reported in the individual assessments, which are based on more 
recent program estimates in some cases. Please see appendix II for details. 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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NAVY and 
MARINE CORPS 
PROGRAM 
ASSESSMENTS 


Most Current and Future Navy 
Programs Report that Less than 20 
Percent of Planned Acquisition Costs 
are Specifically for Software 


The Navy was the Milestone Decision Authority in 2017 for 
22 of its Current and Future Programs in GAO’s Review 

Navy programs 


Percentage of planned acquisition costs 




The milestone decision authority 
for AMDR, CH-53K, CVN 78, DDG 
51, MQ-4C Triton, NGJ-LB, NGJ 
Mid-Band, and VH-92A 
changed from the Under 
Secretary of Defense for 
Acquisition and Sustainment 
(USD(A&S)) to the Navy in 2017. 


The Navy Received One Offer on Solicitations for Just Over Two Thirds of 
its Currently Reported Major Contracts for Development and Procurement 
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Number of programs 


Most Current Navy Programs Report 
that Less than 20 Percent of Work Tasks 
are Software-Related 
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Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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Navy and Marine Corps Program Assessments* 

Page number 

2-page assessments 

Amphibious Combat Vehicle Increment 1.1 (ACV 1.1) 

97 

Air and Missile Defense Radar (AMDR) 

99 

CH-53K Heavy Lift Replacement Helicopter (CH-53K) 

101 

CVN 78 Gerald R. Ford Class Nuclear Aircraft Carrier (CVN 78) 

103 

DDG 1000 Zumwalt Class Destroyer (DDG 1000) 

105 

Ground/Air Task Oriented Radar (G/ATOR) 

107 

F/A-18E/F Infrared Search and Track (IRST) 

109 

Joint Precision Approach and Landing System (JPALS) 

111 

Littoral Combat Ship-Mission Modules (LCS Packages) 

113 

MQ-4C Triton Unmanned Aircraft System (MQ-4C Triton) 

115 

MQ-25 Unmanned Aircraft System (MQ-25 Stingray) 

117 

Next Generation Jammer Mid-Band (NGJ Mid-Band) 

119 

Offensive Anti-Surface Warfare Increment 1 (OASuW Inc 1) 

121 

SSBN 826 Columbia Class Ballistic Missile Submarine (SSBN 826) 

123 

Ship to Shore Connector Amphibious Craft (SSC) 

125 

John Lewis Class Fleet Replenishment Oiler (T-AO 205) 

127 

VH-92A Presidential Helicopter Replacement Program (VH-92A) 

129 

1-page assessments 0 

DDG 51 Arleigh Burke Class Destroyer, Flight III (DDG 51 Flight III) 

131 

Guided Missile Frigate (FFG(X)) 

132 

LHA 8 Amphibious Assault Ship (LHA 8) 

133 

LPD 17 San Antonio Class Amphibious Transport Dock, Flight II (LPD 17 Flight II) [formerly 
LX(R)] 

134 

Next Generation Jammer - Low Band (NGJ-LB) 

135 

P-8A Poseidon, Increment 3 (P-8A Increment 3) 

136 

SSN 774 Virginia Class Submarine Block V (SSN 774 Block V) 

137 


a We abbreviate the following contract types in the individual assessments: cost-plus-award-fee (CPAF), cost-plus-fixed-fee (CPFF), cost-plus-incentive- 
fee (CPIF), firm-fixed-price (FFP), and fixed-price incentive (FPI). 

b DDG 51 Flight III, LFIA 8, LPD 17 Flight II, P-8A Increment 3, and SSN 774 Block V are current major defense acquisition programs that are well into 
production, but we have assessed them in a one-page format because they are planning to introduce new increments of capability. 
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Lead Component: Navy 


Common Name: ACV 1.1 



Source: BAE 


Amphibious Combat Vehicle Increment 1.1 (ACV 1.1) 

The Marine Corps’ ACV is the successor program to the canceled 
Expeditionary Fighting Vehicle. The ACV is intended to transport 
Marines from ship to shore and provide them with improved mobility 
and high levels of protection. The ACV acquisition approach calls for 
three increments of development (1.1, 1.2, and 2.0) and leverages 
work accomplished under the EFV program. We assessed increment 
1.1. Late in our assessment period, in January 2019, the Navy merged 
ACV 1.1 and ACV 1.2 into a single program, the ACV Family of 
Vehicles. 
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Program Essentials 


Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 
Program office: Stafford, VA 

Prime contractors: BAE Systems and 
Land Armaments LP; Science 
Applications International Corporation 
(SAIC) 

Contract types: FPI/FFP/CPFF 
(development—SAIC) 

(development and low-rate initial 
production—BAE Systems and Land 
Armaments LP) 

Software development approach: NA 

Next major milestone: Start of 
operational testing (February 2020) 



First full estimate 
(05/2016) 

Latest 

(07/2018) 

Percentage 

change 

Development 

$821.70 

$696.40 

-15.3% 

Procurement 

$1,091.80 

$1,094.70 

+0.3% 

Unit cost 

$8.18 

$7.64 

-6.6% 

Acquisition cycle 

57 

57 

0.00% 

time (months) 




Total quantities 

240 

240 

0.00% 

Total quantities comprise 36 development quantities and 204 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 



Procurement Quantities 



MFunded to date 
HTo complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Common Name: ACV 1.1 


Lead Component: Navy 

ACV 1.1 Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The ACV 1.1 program has matured its critical 
technologies and stabilized its system design. However, 
the ACV 1.1 contractor is building vehicles prior to 
bringing critical manufacturing processes into statistical 
control—an approach inconsistent with best practices. 

The ACV 1.1’s two critical technologies—the driver 
vision enhancement system and the remote weapon 
station—are fully mature. According to program officials, 
the ACV 1.1 has exceeded its reliability growth targets 
in its most recent tests, excluding failures related to the 
government-furnished remote weapon station. Program 
officials added that they are conducting dedicated 
testing to localize and determine a root cause for the 
remote weapon station failures. 

The ACV 1.1 program released 100 percent of its 
design drawings before the production decision in June 
2018. The Navy approved the program’s entry into low- 
rate initial production with the selected contractor’s 
(BAE Systems) manufacturing readiness below the 
level recommended by the Department of Defense 
guidance and best practices. An assessment made by 
the Defense Contract Management Agency for the ACV 
1.1 production readiness review identified high-risk 
areas such as manufacturing personnel and process 
capability and control. Three months before the 
production decision, the program conducted a risk 
assessment and reported two areas as risks to the 
overall ACV 1.1 production schedule: facilities and 
personnel. Program officials said that, in June 2018, 
BAE Systems submitted a risk mitigation plan that 
addresses these manufacturing risks, and the program 
is monitoring its implementation. 

The program reported that following the production 
decision, the Marine Corps exercised a contract option 
with BAE Systems for 30 low-rate initial production 
vehicles. Officials also reported that in December 2018, 
the Marine Corps exercised another contract option with 
BAE Systems for a second lot of 30 vehicles. 

Other Program Issues 

The Marine Corps selected BAE Systems from two 
competing contractors for ACV 1.1 low-rate initial 
production. During the development phase, two 
contractors designed and produced 16 prototypes each. 
Starting in March 2017, the program conducted 
developmental and live fire testing of the prototypes to 
determine the extent to which they met capability 
requirements such as technical performance, reliability, 
force protection, and system survivability. The Marine 
Corps subsequently conducted an operational 
assessment from January to March 2018 to inform the 
contractor selection and the decision to enter 
production. The operational assessment did not identify 


any systemic problems that would necessitate a major 
redesign, and live fire testing indicated that prototypes 
met key force protection requirements. However, the 
Director, Operational Test and Evaluation, made 
recommendations as a result of the tests, including to 
investigate ways to prevent damage to steering and 
suspension from debris, and to modify the troop 
commander’s station to make it easier to move between 
the hatch and seat. Since the operational assessment, 
the program has held corrective action periods to 
implement minor design modifications in response to 
testing and plans to continue developmental testing into 
December 2019. 

Officials stated that the program has awarded a 
production contract to BAE Systems that includes 
options for the production of the second increment of 
the ACV (ACV 1.2). The primary intended upgrade for 
ACV 1.2 over ACV 1.1 was improved amphibious 
capability, including the ability to self-deploy from an 
amphibious ship and travel to shore. However, the 
program developed and tested these capabilities during 
the development of ACV 1.1. As result, the ACV 
program officials said that they expect the transition to 
the next increment will be relatively seamless. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office noted that, in 
January 2019, it received approval from the Navy to 
merge the ACV programs into a single program, the 
ACV Family of Vehicles. The program office said that 
this merger includes acquiring ACV 1.1 and ACV 1.2 
under a single set of performance requirements, 
developing additional ACV models with different mission 
profiles, and updating acquisition documentation for the 
full-rate production decision review. The program office 
also stated that it is continuing low-rate initial production 
and plans to test production representative vehicles 
beginning in the third quarter of fiscal year 2019. 
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Lead Component: Navy 


Common Name: AMDR 



Source: © 2015 Raytheon Company 


Air and Missile Defense Radar (AMDR) 

The Navy's AMDR is a next-generation radar program supporting 
surface warfare and integrated air and missile defense. The Navy 
expects AMDR’s radar—known as AN/SPY-6(V)1—to provide 
increased sensitivity for long-range detection to improve ballistic 
missile defense against advanced threats. The program office is also 
developing a radar suite controller that will interface with an upgraded 
Aegis combat system to provide integrated air and missile defense for 
DDG 51 Flight III destroyers. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 


First full estimate 

Latest 

Percentage 

Program office: Washington Navy Yard, 
DC 

Prime contractor: Raytheon 


(10/2013) 

(06/2018) 

change 

Development 

$2,029.70 

$1,911.10 

-5.8% 

Contract types: CPIF (development) 

Procurement 

$4,197.80 

$3,634.90 

-13.4% 

FPI (low-rate initial production) 

Unit cost 

$284.50 

$253.51 

-10.9% 

Software development approach: Agile 
development 

Acquisition cycle 
time (months) 

156 

161 

+3.2% 

Next major milestone: First production 




radar delivery (April 2020) 

Total quantities 

22 

22 

0.00% 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

S207.11 _ $1, 70 3,9 6 


o 


$2.849.31 


Procurement 

$785.57 



Procurement Quantities 
4 




Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Lead Component: Navy 


Common Name: AMDR 


AMDR Program 

Technology Maturity and Design Stability 

The program office reported that AMDR’s four critical 
technologies are mature—although we disagree—and 
that the system design is stable. Since our 2018 
assessment, the program office has further 
demonstrated the radar system’s performance and 
capabilities through live testing and simulation. 

However, based on industry best practices, the program 
cannot fully demonstrate all critical technologies until 
the Navy tests them in their realistic, at-sea 
environment with the Aegis combat system. According 
to the AMDR program schedule, such testing will occur 
in 2023 during operational testing with a DDG 51 Flight 
III ship. Until the Navy completes this testing, the 
program’s design stability remains at risk for disruption. 
Specifically, any performance deficiencies the Navy 
discovers during at-sea testing could require it to revise 
existing design drawings to remedy issues. 

As part of developmental testing, the program office 
tested a full-scale, single-face radar array at the Navy’s 
Pacific Missile Range Facility (PMRF) beginning in 
September 2016. The program office successfully 
completed several live ballistic missile defense, anti-air, 
and anti-surface warfare tests. However, in March 2018, 
the array failed a ballistic missile test because of a 
defective software update that caused the array to stop 
tracking a live target. Officials said a software update 
corrected the issue and they verified the array’s 
performance through a successful retest in January 
2019. Officials said the single-face array, originally 
scheduled to support Aegis combat system equipment 
testing, will undergo additional testing at PMRF through 
2019. As a result, the Navy has revised the acquisition 
schedule and will instead divert the delivery of a new 
array to support land-based Aegis combat system 
equipment testing sometime in 2019. 

The program has completed software development to 
support core radar capabilities and will continue to 
develop radar updates to support system 
improvements, cybersecurity, and combat system 
integration through 2021. In parallel to the radar’s 
software development, significant software development 
remains to integrate AMDR with the Aegis combat 
system. Program officials said this software 
development must complete before both systems can 
be fully integrated and tested. While the Navy plans to 
test the radar and initial Aegis combat system software 
at a land-based site, the Navy will not test the radar and 
final Aegis combat system until both are installed on the 
lead ship. Any issues identified after the systems are 
installed on the lead ship could require retrofits to the 
radar or ship. 


Production Readiness 

Nearly 18 months after entering production, the 
program has not demonstrated that all of its critical 
manufacturing processes are in statistical control. The 
program reported that it exercised a contract option for 
the fourth low-rate initial production unit in April 2018 
and was authorized to procure five additional low-rate 
production units in February 2019. However, in August 
2018, the contractor reported early cost growth and 
schedule variance for the first three low-rate production 
units because of increased material costs and other 
production delays. Officials said the delays are partly 
due to a problem with a digital receiver component, 
which the contractor is testing. As a result, contractor 
delivery of the first production radar is at risk of delay 
from December 2019 to April 2020. 

The AMDR program office plans to procure more than 
two-thirds of its 22 total radars prior to completing 
operational testing. The Navy deliberately planned for 
AMDR to begin production prior to the start of Aegis 
upgrade software development to allow time for key 
radar technologies to mature and for the design to 
stabilize. However, this concurrency means any 
deficiencies identified during combat system integration 
or operational testing may lead to retrofitting after 
production is underway or complete for many of the 
radars. Any required retrofitting is likely to increase 
program costs or delay radar deliveries. 

Other Program Issues 

DOD’s Director, Operational Test and Evaluation 
(DOT&E) has yet to approve the AMDR Test and 
Evaluation Master Plan. DOT&E stated that the 
proposed test approach for the AMDR and DDG 51 
Flight III programs does not provide realistic operational 
conditions without the use of an AMDR- and Aegis- 
equipped unmanned self-defense test ship. Because 
the Navy has elected not to request funds for a test 
ship, DOT&E and the Navy are revising the DDG 51 
Flight III operational test strategy to include AMDR 
operational requirements and an updated simulation 
strategy. DOT&E cautioned, however, that DDG 51 
Flight Ill’s self-defense and survivability capabilities will 
not be fully known until the program completes 
operational testing. 

Program Office Comments and GAO Response 

We provided a draft of this assessment to the program 
office for review and comment. In its comments, the 
program office disagreed with our assessment of the 
program’s technology maturity, stating that combat 
system testing is not required to demonstrate mature 
radar technologies since the technologies have been 
tested and proven at the land-based PMRF site. We 
disagree. The PMRF site does not provide a realistic, 
at-sea environment to test the fit and function of the 
radar and combat system on a ship. 
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Lead Component: Navy 


Common Name: CH-53K 



Source: U.S. Navy. 


CH-53K Heavy Lift Replacement Helicopter (CH-53K) 

The Marine Corps' CH-53K heavy-lift helicopter is intended to transport 
armored vehicles, equipment, and personnel to support operations 
deep inland from a sea-based center of operations. The CH-53K is 
expected to replace the legacy CH-53E helicopter and provide 
increased range and payload, survivability and force protection, 
reliability and maintainability, and coordination with other assets, while 
reducing total ownership costs. 
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Program Essentials 

Milestone decision authority: Navy 
Program office: Patuxent River, MD 
Prime contractor: Sikorsky Aircraft 
Contract type: CP IF 
(development) 

FFP/CPFF 

(engine development) 

FPI/CPFF/FFP (LRIP) 

Software development approach: Mixed 

Next major milestone: Start of operational 
testing (May 2021) 


Program Performance (fiscal year 2019 dollars in millions) 


First full estimate 
(12/2005) 

Latest 

(07/2018) 

Percentage 

change 

Development 

$4,902.9 

$7,465.8 

+52.3% 

Procurement 

$13,635.6 

$21,366.1 

+56.7% 

Unit cost 

$118.84 

$144.23 

+21.4% 

Acquisition cycle 

117 

187 

+59.8% 

time (months) 




Total quantities 

156 

200 

+28.2% 


Total quantities comprise 6 development and 194 procurement quantities. Latest initial capability date reflects a 16- 
month delay from the July 2018 estimate. The program has not yet determined any associated cost changes. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$824.03 ^ $6,641.77 


a 


$20,094.85 


Procurement 

$1,271.24 


a 


Procurement Quantities 
6 


-G 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


| Funded to date 
| To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Common Name: CH-53K 


Lead Component: Navy 

CH-53K Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The CH-53K program entered production in March 2017 
with mature critical technologies and a stable design, 
but without demonstrated production processes. Due to 
a number of technical deficiencies identified through 
developmental testing, since last year’s assessment, 
the program office has delayed the start of operational 
testing by almost 2 years, to May 2021, and postponed 
initial operational capability by 20 months, to September 
2021. 

The program office identified two critical technologies— 
the main rotor blade and the main gearbox—for CH- 
53K. Although,the program office reported that both 
critical technologies are mature, there are technical 
issues with the main gear box causing low service life 
projections. The program office also noted that while 
there are parts shortages with the main gearbox,the 
supplier has recently improved its manufacturing 
processes in an effort to reduce the backlog of needed 
parts. It is too soon to tell if this will reduce the parts 
shortages. 

The program office also reported that it has released 99 
percent of CH-53K design drawings, which constitutes a 
stable design. Nonetheless, recent deficiencies 
discovered in developmental testing may require the 
program to undertake design changes that could disrupt 
that design stability.Developmental tests have revealed 
high engine bay temperatures, which could cause 
exhaust gas to enter the aircraft fuselage during 
operations. The program extended its developmental 
testing into January 2021. Program officials stated that 
all technical problems likely will not be addressed 
before operational testing begins in May 2021. The 
program office is currently amending its test plan and is 
considering different scenarios to address this situation. 

Delivery of production aircraft is scheduled for the fourth 
quarter of fiscal year 2020. The program has yet to 
demonstrate that its critical manufacturing processes 
are in statistical control—an approach inconsistent with 
best practices. Since our 2018 assessment, the 
program delayed award of a planned second low-rate 
initial production contract for four aircraft. The program 
planned to award this contract in March 2018, but 
changes to the program’s test plan have caused the 
program to delay this award. 

Further, the CH-53K contractor has delivered to the 
Navy only three of six planned system demonstration 
test articles. The contractor was scheduled to deliver 
the fourth test article to the Navy in the first quarter of 
fiscal year 2019. However, the contractor used the 
fourth test article to supply parts for the first three due to 
parts shortages. The contractor now plans to deliver the 


fourth aircraft by December 2020. According to the 
program office, the first two test articles are currently 
being used for developmental testing while the third was 
delivered to the Marines for logistics demonstration 
projects. The program plans to use the first four test 
articles in operational testing. However, the first three 
will need to be modified first, which is not expected to 
be completed by the start of testing. If the fourth aircraft 
delivery is delayed again, the Navy will have to delay 
operational testing until the aircraft are ready. 

Other Program Issues 

The program office has increasingly discovered 
software deficiencies over the past year. Specifically, 
developmental testing showed a failure in the software 
to detect the transition from ground to flight causing 
increased safety concerns. The program is currently 
working to solve this problem through additional 
software development. The program office stated they 
generally revised the software plan so they could be 
more flexible in addressing software deficiencies. The 
program office also reported increased risk in 
completing additional software development efforts to 
meet cyber security needs. According to the office of 
the Director, Operational Test and Evaluation, the 
program updated its test and evaulation plan to 
emphasize cyber security test strategies. 

In addition, the contractor moved the CH-53K assembly 
line from West Palm Beach, Florida, to Stratford, 
Connecticut, in June 2018. The final two test articles will 
be produced in Connecticut, but the program office 
does not expect the relocation to affect production. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that, 
because of technical issues during developmental 
testing, the program has completed test events slower 
than planned, which has affected schedule and cost. 
The program will miss four key baseline dates, and the 
program is establishing a new schedule baseline. The 
program office noted the planned dates for initial 
capability, end operational test, and full-rate decision 
are tentative pending a revised acquisition program 
baseline. The program’s top priorities for the start of 
operational testing are to resolve the remaining 
technical issues and complete airworthiness certification 
testing. Specifically, the program office stated that it has 
prioritized the flight test plan to address the top 
technical issues, correct other technical deficiencies, 
and deploy the fleet as planned in 2024. 
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Lead Component: Navy 


Common Name: CVN 78 



Source: U.S. Navy. 


CVN 78 Gerald R. Ford Class Nuclear Aircraft Carrier 
(CVN 78) 

The Navy developed the CVN 78 (or Ford Class) nuclear-powered 
aircraft carrier to introduce new propulsion, aircraft launch and 
recovery, and survivability capabilities to the carrier fleet. The Ford 
Class is the successor to the Nimitz Class aircraft carrier. Its new 
technologies are intended to create operational efficiencies while 
enabling a 33 percent increase in sustained operational aircraft flights 
over legacy carriers. The Navy also expects the new technologies to 
enable Ford Class carriers to operate with reduced manpower. 
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Program Essentials 


Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 
Program office: Washington, DC 
Prime contractor: Huntington Ingalls 
Industries 

Contract types: CPFF/CPIF (CVN 79 
construction preparation); CPFF/FPI 
(CVN 79 detail design and construction); 
CPFF/undefinitized (CVN 80 advance 
procurement); FPI (CVN 80/81 detail 
design and construction) 

Next major milestone: Initial operational 
capability (July 2019) 

Software development approach: NA 


First full estimate 
(04/2004) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$5,378.60 

$6,096.50 

+13.3% 

Procurement 

$34,456.50 

$42,568.60 

+23.5% 

Unit cost 

$13,278.37 

$12,213.29 

-8.0% 

Acquisition cycle 

137 

183 

+33.6% 

time (months) 




Total quantities 

3 

4 

+33.3% 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 



Procurement Quantities 



Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


Complete basic and functional design to include 100 
percent of 3D product modeling 


[Knowledge attained, Q Knowledge not attained, ■information not available, Not applicable 


Detail Design 
Contract Award 



■ Funded to date 

■ To complete 
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Common Name: CVN 78 


Lead Component: Navy 


CVN 78 Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The Navy accepted delivery of the lead ship, CVN 78, in 
May 2017 despite challenges related to immature 
technologies and struggles to demonstrate the reliability 
of mature systems. The Navy reports that 10 of the Ford 
Class’s 12 critical technologies are fully mature—the 
advanced arresting gear (AAG) and one of the ship’s 
missile systems are not yet mature. The advanced 
weapons elevators are among the systems deemed 
mature by the Navy; however, according to Navy 
officials, only 2 of the 11 elevators installed on the ship 
can bring munitions to the flight deck—a key element of 
operational flights. The shipbuilder is working to correct 
the system during its first post-delivery maintenance 
period, now scheduled to end in October 2019, and the 
Navy plans to create a land-based site to test the 
elevators, which will come at an additional cost. 

Shipboard testing is ongoing for several critical systems 
and could delay future operational testing. Those 
systems include the electromagnetic aircraft launch 
system (EMALS), AAG, and dual band radar (DBR). 
Although the Navy is testing EMALS and AAG on the 
ship with aircraft, the reliability of those systems 
remains a concern. If these systems cannot function 
safely, CVN 78 will not demonstrate it can rapidly 
deploy aircraft—a key requirement for these carriers. 
Recent shipboard testing revealed that the Navy is 
struggling to get DBR to operate as planned. Moreover, 
DBR poses a greater radiation hazard to personnel and 
systems on an aircraft carrier than the Navy anticipated, 
which could restrict certain types of flight operations. 

The remaining challenges the Navy faces in maturing 
CVN 78’s critical technologies could lead to their 
redesign or replacement on later ships. This would 
include CVN 79, which is currently 55 percent complete, 
as well as the third and fourth ships, CVNs 80 and 81. 
CVN 79 repeats the CVN 78 design with some 
modifications and replaces DBR with the Enterprise Air 
Surveillance Radar (EASR), which is in development. 
The Navy does not identify this new system as a critical 
technology in the Ford Class program because it 
derives from the pre-existing Air and Missile Defense 
Radar (AMDR) program. However, EASR is a different 
size and performs a different mission than the AMDR 
systems, which are designed for destroyers. Therefore, 
EASR may still require design and development efforts 
to function on the carrier.The Navy plans to procure two 
EASR units for CVNs 79 and 80 and install the CVN 79 
unit during that ship’s second phase of delivery. CVNs 
80 and 81 will repeat the design of CVN 79. 

Other Program Issues 

CVN 78’s procurement costs increased by 23 percent 
over its initial cost cap and as a result of continuing 
technical deficiencies, the Navy may still require more 


funding to complete this ship. The Navy increased the 
current $12.9 billion cost cap for CVN 78 by $120 
million in May 2018 to account for additional post¬ 
delivery work, but added work and cost changes may 
result in an additional cost increase. 

Costs for CVN 79 are also likely to increase as a result 
of optimistic cost and labor targets, putting the ship at 
risk of exceeding its $11.4 billion cost cap. The CVN 79 
cost estimate assumes unprecedented construction 
efficiency—labor hours will be 18 percent lower than 
CVN 78. However, our analysis shows the shipbuilder is 
not meeting this goal and is unlikely to improve 
performance enough to meet cost and labor targets. 

Congress raised the cost cap for CVN 80 and later 
ships to $12.6 billion and approved the Navy’s plans to 
buy two carriers—CVNs 80 and 81—at the same time, 
based on the shipbuilder’s estimate that this strategy 
will save the Navy over $2 billion. However, it is unclear 
whether the Navy can meet this cost cap, even with the 
estimated savings from a two-ship buy, because it 
assumes further reductions in subsystem costs, 
construction change orders, and labor hours. The Navy 
projects a further reduction in labor hours compared to 
CVN 79—about 25 percent fewer labor hours than CVN 
78—will contribute to cost savings for these ships. 

The program office indicated that it does not separately 
track or report information on software development to 
integrate the various subsystems of the ship. These 
subsystems include CVN 78’s combat control systems, 
which rely on integrating systems through software 
intensive development. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that, in 
July 2018, CVN 78 entered a year-long maintenance 
period. It also said that, as of February 2019, two 
advanced weapons elevators are operating, and it 
continues to improve developmental system reliability. 

The program also stated that, with CVN 79 construction 
55 percent complete, shipbuilder cost performance 
remains stable, but slightly below the level needed to 
achieve production labor hour reduction targets. The 
program stated that the shipbuilder continues to work 
through the effects of material shortfalls that disrupted 
performance. The program said that the Navy plans to 
deliver a complete, deployable ship as scheduled and 
within the cost cap to maintain an 11-carrier fleet. 

The program office also stated that the Navy awarded 
the CVN 80/81 procurement contract in January 2019 
and expects to save $4 billion, compared to if it had 
purchased each ship individually. According to the 
program, the contract limits the Navy’s liability and 
incentivizes the shipyard’s best performance. 
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Lead Component: Navy 


Common Name: DDG 1000 



Source: U S Navy 


DDG 1000 Zumwalt Class Destroyer (DDG 1000) 

The DDG 1000 destroyer is a multi-mission surface ship initially 
designed to provide advanced capability to support forces on land. 
DDG 1000 class ships feature a stealth design, integrated power 
system, and total ship computing environment. The Navy adopted a 
phased acquisition strategy, which separates delivery and acceptance 
of hull, mechanical, and electrical (HM&E) systems from combat 
system activation and testing. The Navy has recently changed DDG 
1000’s primary mission from land attack to offensive surface strike. 
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Program Essentials Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 
Program office: Washington, DC 


First full estimate 
(01/1998) 

Latest 

(06/2018) 

Percentage 

change 

Prime contractors: General Dynamics 
Bath Iron Works; BAE Systems; 

Development 

$2,550.53 

$11,876.75 

+365.7% 

Huntington Ingalls Industries; Raytheon 

Procurement 

$36,414.13 

$13,538.91 

-62.8% 

Contract types: FPI/FFP/CPFF (ship 
construction); FPI/CPFF(advanced gun 

Unit cost 

1,217.65 

8,471.89 

+595.8% 

systems equipment); CPFF/CPAF 

Acquisition cycle 

128 

273 

+113.3% 

(mission systems equipment) 

Software development approach: 

Mixed 

time (months) 

Total quantities 

32 

3 

-90.63% 


Next major milestone: Lead-ship final 
delivery (May 2019) 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 


$576.80 



Procurement Quantities 



Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Complete basic and functional design to include 100 
percent of 3D product modeling 


Detail Design 
Contract Award 


[Knowledge attained, Q Knowledge not attained, ■information not available, Not applicable 



■ Funded to date 

■ To complete 
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Lead Component: Navy 

DDG 1000 Program 


Common Name: DDG 1000 


Technology Maturity and Design Stability 

The DDG 1000 program has fully matured most, but not 
all, of its nine current critical technologies and reports a 
stable design. According to the Navy, the fire 
suppression system, hull form, deckhouse, power 
system, and undersea warfare suite technologies are all 
mature. At the same time, the vertical launch system, 
infrared signature, multi-function radar, and total ship 
computing environment technologies each continue to 
approach maturity. The Navy expects to fully mature 
these systems as it completes ship construction, 
certification, and testing over the next 2 years. 

The program originally had 12 critical technologies, but 
in the last several years, the Navy removed three, 
including two technologies associated with the 
advanced gun system—the projectile and the gun— 
because of the projectile’s high cost per round. The 
Navy planned to rely on these munitions for precision 
fires and offensive operations. Following an evaluation 
of five other munition options, the Navy determined that 
no viable replacement, guided or unguided, was 
feasible. As a result, the guns will remain inoperable on 
the ships for the foreseeable future. Lastly, the Navy will 
use a modified multi-function radar in place of a volume 
search radar, which the Navy removed from the class. 

As we have previously reported, the Navy and its 
shipbuilders had not stabilized DDG 1000’s design by 
lead ship fabrication start in 2009—an approach 
inconsistent with best practices. This approach 
contributed to numerous design changes after the 
fabrication start and significant cost increases and 
schedule delays. Nearly 10 years later, development 
and shipboard testing of technologies continues, each 
of which could lead to discovery that could disrupt the 
design stability the Navy currently claims. 

The Navy plans to complete software development for 
the class in September 2020—a delay of 24 months 
since our 2018 assessment. As a result, the Navy has 
had to delay some testing. Also that month, the program 
plans to complete its cyber security vulnerability 
evaluation along with the remainder of a 2-year regimen 
of certifications and several different tests. The Navy 
expects this regimen to demonstrate the full 
functionality of the ship’s systems. 

Production Readiness 

The DDG 1000 shipbuilder is approaching completion of 
the hull, mechanical, and electrical (HM&E) systems for 
all three ships of the class. Shipbuilder delivery of the 
lead ship's HM&E occurred 18 months behind schedule, 
in part because of problems completing electrical work 
associated with the ship’s power system. The 
shipbuilder also experienced problems completing the 
power system for DDG 1001, the second ship in the 
class. Following sea trials, the Navy inspected one of 


the ship’s main turbine generators and found that the 
generator was damaged by a woodscrew. The damage 
was extensive enough that the Navy chose to replace 
the engine and send it for repair. Officials report that the 
shipbuilder delivered the ship in April 2018 and the 
Navy replaced the engine in September 2018 at its 
expense. 

The Navy has scheduled DDG 1000’s final delivery, 
including HM&E and combat systems, for May 2019. 
The Navy has scheduled DDG 1001’s final delivery to 
follow in September 2020. However, the Navy is still 
working to correct serious deficiencies that its Board of 
Inspection and Survey has identified on both ships. 
Specifically, the board found over 320 serious 
deficiencies when the shipbuilder delivered DDG 1000’s 
HM&E in May 2016, and 246 serious deficiences after 
the Navy conducted acceptance trials for DDG 1001 in 
January and February 2018. This increases the 
likelihood that the ship will not be fully capable and 
sustainable when provided to the fleet. 

To limit further delays to DDG 1000 and DDG 1001 
construction, the Navy has authorized its shipbuilder to 
take parts from DDG 1002—the third and final ship of 
the class, which is under construction. The Navy does 
not yet know the full extent to which these actions will 
delay DDG 1002’s construction schedule, but stated 
that these parts typically can be borrowed and replaced 
without causing a delay. The Navy has scheduled the 
ship’s HM&E delivery in March 2020 followed by final 
delivery in September 2022. 

Other Program Issues 

In a January 2018 decision memorandum, the Navy 
changed DDG 1000’s primary mission from land attack 
to offensive surface strike. Navy officials are in the 
process of determining the operational concept for the 
ship within its new mission. The Navy has yet to 
establish testing plans to evaluate these future mission 
sets. According to Navy officials, the Navy’s planned 
modifications to support the new mission will cost about 
$1 billion, from non-acquisition accounts. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office also stated that it 
is making good progress delivering the Zumwalt class. 
The Navy said that, since our assessment, DDG 1000 
completed combat systems availability, combat tests 
are underway, and final delivery is now planned for 
September 2019. The program office also said that 
DDG 1001 started combat systems availability in April 
2019, and DDG 1002 is 84 percent constructed. The 
program office further noted that future addition of new 
systems onto Zumwalt-class ships will provide offensive 
fire capabilities. 
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Lead Component: Navy 


Common Name: G/ATOR 




Power Equipment 
Group (PEG) 





Radar Equipment Communications 

Group (REG) Equipment Group (CEG) 


Ground/Air Task Oriented Radar (G/ATOR) 

The Marine Corps' G/ATOR is a three-dimensional, short-to-medium 
range, multi-role radar designed to detect, identify, and track threats 
such as incoming cruise missiles, rockets, and artillery. It will replace 
five legacy radars. G/ATOR is being acquired in blocks, with later 
blocks focused on software upgrades. We assessed Block 1, which 
has an air defense and surveillance role, and have made observations 
on Block 2, which will determine enemy firing positions and point of 
impact for incoming fire. 


Source: U.S. Marine Corps. 
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Program Essentials 

Milestone decision authority: Navy 
Program office: Quantico, VA 
Prime contractor: Northrop Grumman 
Contract type: FPI/CPFF/FFP (low-rate 
initial production) 

Software development approach: 

Incremental 

Next major milestone: Initial operational 
capability for Block 2 (February 2019) 


o 


Procurement 

$706.03 


$1.222.53 


Procurement Quantities 

O 


Program Performance (fiscal year 2019 dollars in millions) 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$132.45 ^ _ $1.034,59 



First full estimate 
(08/2005) 

Latest 

(07/2018) 

Percentage 

change 

Development 

$396.50 

$1,167.00 

+194.4% 

Procurement 

$1,244.90 

$1,928.60 

+54.9% 

Unit cost 

$25.65 

$68.79 

+168.2% 

Acquisition cycle 
time (months) 

61 

150 

+145.9% 

Total quantities 

64 

45 

-29.7% 


Attainment of Product Knowledge 



As of January 2019 


Status at 

Current Status 

Resources and requirements match 

Development 

Start 



Funded to date 
| To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Common Name: G/ATOR 


Lead Component: Navy 

G/ATOR Program 

Technology Maturity, Design Stability, and 
Production Readiness 

According to the G/ATOR program office, the radar’s six 
critical technologies are mature, design is stable, and 
production processes are in control as the program 
approaches its full-rate production decision in April 
2019. We were unable to assess G/ATOR’s technology 
maturity at development start because the program did 
not generate the type of information needed for such an 
assessment. 

The program office stated that the system is on track to 
demonstrate all of its key performance requirements; 
however, system reliability—while improved—remains a 
concern due mostly to software issues. The program’s 
operational assessment, conducted from March to May 
2018, raised a series of reliability and usability 
deficiences that the program is working to address. 

Program officials stated that the Marine Corps decided 
to wait to field the Block 2 radar until 2019 due to the 
issues raised during the operational assessment. 

During that assessment, Marine Corps testers noted 
that while Block 2 radar offered greater detection 
ranges than current radars, those advantages were 
offset by reliability concerns, among other issues. 
According to program officials, several of the issues 
raised in the assessment resulted from the Marine 
testers being inexperienced and unfamiliar with 
operating the radar system. In other cases, they were 
due to known software problems, which the program 
said that it has addressed. The operational assessment 
resulted in eight recommendations, and program 
officials stated they have taken actions to address them. 

The program completed operational testing, which 
assessed the effectiveness, suitability, and survivability 
of the Block 1 and Block 2 radars, in October and 
December 2018, respectively. The testing report for 
Block 1 found it to be operationally effective, suitable, 
and survivable. However, the testing report for Block 2 
was not complete at the time of our review. 

The G/ATOR program is well into low-rate initial 
production and its production processes have matured. 
At the program’s March 2014 production decision, the 
contractor had demonstrated G/ATOR production 
processes to the DOD recommended level, but had not 
brought them into statistical control, which was 
inconsistent with best practices. In 2018, the program 
completed a production readiness review to support full- 
rate production and reported that its production 
processes are now in statistical control. The Marine 
Corps accepted delivery of eight production radars in 
2018 as scheduled, and the contractor delivered the 
first two radars incorporating new semiconductor 
technology about one month ahead of schedule. 


Beginning with radars produced in 2016, the program 
upgraded G/ATOR’s “transmit/receive modules”—key 
components that process signals from and to the 
radar—to a new, but mature gallium nitride (GaN) 
semiconductor technology. The GaN semiconductors fit 
inside the G/ATOR system the same way as the older 
gallium arsenide (GaAs) semiconductors they replaced. 
Program officials expect the GaN technology to 
achieve better performance with higher reliability at 
lower cost by reducing the number of modules required. 
The program office has budgeted $19.8 million from 
fiscal years 2022 through 2024 to refurbish the first six 
GaAs radars with GaN technology, and to bring all 15 
low rate initial production systems up to current full-rate 
production configuration. The Marine Corps also 
continues to fund initiatives to add new capabilities to 
the radar. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
stated that the G/ATOR program continues to meet its 
cost, schedule, and performance commitments 
following establishment of a new acquisition program 
baseline in 2010. The program office stated that it has 
accepted delivery of eight low-rate initial production 
systems and has seven additional systems under 
contract. According to the program office, G/ATOR has 
demonstrated all Block 1 and Block 2 key performance 
requirements. The program office stated that G/ATOR 
will meet initial operational capability with two Block 1 
systems fielded in 2018 and four Block 2 systems 
fielded in 2019. According to the program office, the 
program is on track for a full-rate production decision in 
2019 as well as delivery of the required capability to the 
warfighter. The program office also stated that 
G/ATOR's operational availability has exceeded the 
system’s requirement and early software quality 
challenges addressed. 
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Lead Component: Navy 


Common Name: IRST 



Source: U S Navy 


F/A-18E/F Infrared Search and Track (IRST) 

The Navy is developing an infrared search and track (IRST) sensor to 
integrate onto the F/A-18E/F fuel tank. IRST will allow F/A-18s to 
detect and track objects from a distance in environments where radar 
is not effective. The Navy is acquiring IRST with an evolutionary 
acquisition approach. It used existing IRST systems to develop Block I 
and is implementing Block II to deliver an improved sensor, upgraded 
processor, and additional software. We assessed IRST Block II. 
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Program Essentials 

Milestone decision authority: Navy 
Program office: Patuxent River, MD 

Program Performs 

ince (fiscal year 2019 dollars in millions) 


First full estimate 
(02/2017) 

Latest 

(12/2018) 

Percentage 

change 

Prime contractor: Boeing 





Contract type: CPIF (Block II 

Development 

$899.00 

$941.00 

+4.7% 

development), FPIF (Block II low-rate 

Procurement 

$1,354.00 

$1,403.20 

+3.6% 

production) 

Unit cost 

$12.59 

$13.55 

+7.7% 

Software development approach: 





Mixed 

Acquisition cycle 

123 

123 

0.00% 

Next major milestone: Start of 

time (months) 




operational testing (August 2020) 

Total quantities 

179 

173 

-3.4% 


The Navy approved a new acquisition program baseline in December 2018. Total quantities comprise 3 
development quantities and 170 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions 


$357.97 


Development 



Procurement 


$1.058.48 



Procurement Quantities 



MFunded to date 
MTo complete 


Attainment of Product Knowledge 

As of January 2019 (Block II activities only) 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Common Name: IRST 


Lead Component: Navy 

IRST Program 

Technology Maturity, Design Stability, and 
Production Readiness 

IRST Block II entered low-rate production in December 
2018 with mature critical technologies, but an unstable 
design and undemonstrated critical manufacturing 
processes. The IRST program’s only critical technology 
is its “passive ranging algorithm” tracking software. The 
program expects this software to track targets better 
than legacy system software. In a July 2017 technology 
maturation assessment, an independent review panel 
assessed the software as fully functional in an 
operational environment. 

Although in production, IRST does not yet have a stable 
design. At the time of the program’s November 2018 
critical design review, the contractor had released about 
77 percent of Block II design drawings—a level that falls 
short of the 90 percent recommended by best practices 
as a mark of design stability. As of January 2019, the 
program had increased its drawing releases to about 82 
percent of planned total. The program also has yet to 
test a system-level integrated prototype. According to 
program officials, they have tested hardware and 
software in configurations similar to those planned for 
the Block II design—but short of a prototype—to 
validate design efforts for a future Block II system-level 
prototype. Our prior work has shown that carrying a 
stable design into production reduces the risk of costly 
design changes to already-produced systems. 

The program also entered production without 
demonstrating critical manufacturing processes both on 
a pilot production line and within statistical process 
controls—inconsistent with best practices. Program 
officials said they will review or modify Block I 
manufacturing processes, which they consider mature, 
that are used for building Block II prototype hardware, 
with deliveries starting in late fiscal year 2019. The 
program also plans for IRST development and 
production to overlap in order to achieve the 
accelerated initial operational capability. As we have 
found in our evaluations of numerous other DOD 
programs, this type of concurrent approach increases 
risk of program cost increases and schedule delays. 

Officials reported that in December 2018, the program 
awarded an undefinitized contract action, under fixed- 
price incentive firm target terms, for low-rate initial 
production of six Block II units. According to the 
program office, this contract action supports F/A-18 
operations by accelerating IRST’s initial operational 
capability by over 2 years, from January 2024 to 
September 2021. The program plans to award two 
additional contracts—one in fiscal year 2020 for 12 
units and another in fiscal year 2021 for 25 units—for 
low-rate initial production. Consequently, the program 
intends to acquire over 28 percent (43 of 152) total 
production quantities through low-rate initial production. 
If a program’s low-rate initial production quantity 


exceeds 10 percent of the total production quantity, the 
program must provide a rationale for these quantities in 
a report to Congress. For IRST, the Navy’s rationale 
hinges on establishing an initial production base and 
allowing an orderly increase in production leading to 
full-rate production. 

Other Program Issues 

A May 2014 report from DOD’s Inspector General found 
the the Navy had inappropriately planned to use IRST 
procurement funds to fund Block II development 
activities and recommended the funds be 
reprogrammed. Following the reprogramming, the Navy 
reclassified IRST as a major defense acquisition 
program based on its adjusted total research, 
development, technology, and evaluation costs. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program had no 
comments on the assessment. 
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Lead Component: Navy 


Common Name: JPALS 



Source: U S. Navy 



Joint Precision Approach and Landing System (JPALS) 

JPALS is a program to develop a Global Positioning System (GPS)- 
based aircraft landing system that will allow aircraft such as the F-35 
Lightning II and the MQ-25 Unmanned Aircraft System to operate from 
aircraft carriers and amphibious assault ships. With JPALS, the Navy 
intends to provide a reliable, sea-based precision approach and 
landing capability that is effective in adverse weather conditions. 
JPALS functionality is primarily software-based, although it will also 
feature off-the-shelf hardware such as antennas and racks. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 
Program office: Lexington Park, MD 


First full estimate 
(07/2008) 

Latest 

(06/2018) 

Percentage 

change 

Prime contractor: Raytheon 

Contract type: CPIF (development) 

Development 

$886.90 

$1,494.70 

+68.5% 

Software development approach: 

Procurement 

$238.80 

$415.10 

+73.9% 

Mixed 

Unit cost 

$30.63 

$58.11 

+89.7% 

Next major milestone: Low-rate initial 
production (March 2019) 

Acquisition cycle 
time (months) 

77 

146 

+89.6% 


Total quantities 

37 

33 

-10.8% 


Total quantities comprise 10 development quantities and 23 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$240.87 ^ _ $1.253,88 


O 


Procurement 

MAAAL _ $ 0-62 


o 


Procurement Quantities 




Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
| To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


|Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Lead Component: Navy 

JPALS Program 


Common Name: JPALS 


Technology Maturity and Design Stability 

Both of JPALS’s two critical technologies are 
approaching maturity, and the program has released 
100 percent of its design drawings, which corresponds 
with a stable design. However, as the program 
continues to mature its critical technologies through 
testing, the program may need to revise its design 
drawings to accommodate these changes, which could 
compromise design stability. 

JPALS originally entered system development in July 
2008 and held a critical design review (CDR) in 
December 2010, but the design later proved unstable. 
The program proceeded with development and 
accepted delivery of eight prototypes. As JPALS 
approached its original production decision in 2013, 
other military departments and civilian agencies decided 
to continue using their current landing systems instead 
of investing their resources in JPALS. As a result, the 
Navy restructured the JPALS program from seven 
increments to one. 

Because of the restructure, the Navy revised its 
schedule and milestones and conducted a new system- 
level preliminary design review in March 2016, a new 
development start in June 2016, and a new CDR in May 
2017. Because the program repeated these three 
events, our attainment of product knowledge table 
assesses the program’s knowledge at its original 
development start and original CDR events, which 
formed the basis for the program’s original business 
case. This methodology is consistent with how we have 
previously assessed JPALS and other programs that 
have repeated key program events. 

In June 2016, Navy leadership authorized the 
restructured JPALS program to enter the engineering 
and manufacturing development phase. The program 
office reported that it awarded a contract in September 
2016 to upgrade the eight original prototypes, as well as 
to procure two additional prototypes for developmental 
testing. The contractor delivered these prototypes 
during the second quarter, of fiscal year 2018, 
according to program officials. Both the new and 
upgraded prototypes are intended to be production 
representative. According to program officials, these 
prototypes will allow the program to demonstrate the 
JPALS critical technologies in a realistic environment, 
which the program plans to do prior to entering 
production. 

Production Readiness 

JPALS does not have any critical manufacturing 
processes, according to the program, because the 
hardware is primarily off-the-shelf. In December 2017, 
the Navy approved the JPALS program to procure the 
entirety of its 23 production units through low-rate initial 
production because it anticipates cost savings through 


shortening the procurement schedule. As a result, the 
program updated its baseline in March 2018 to reflect 
that it would not execute a full-rate production decision. 
Program officials reported that they completed an 
operational test readiness review in April 2018 and 
attained early operational capability with their 
prototypes in June 2018 to support F-35 Lightning II 
operational testing. For fiscal year 2018, program 
officials reported a combined total of 78 aircraft 
approaches for integrated and operational testing. They 
also stated the program successfully completed its 
production readiness review in December 2018 ahead 
of the planned March 2019 low-rate initial production 
decision. 

Other Program Issues 

Because JPALS is GPS-based, it will need to be 
compliant with with any updates to GPS systems, such 
as the integration of M-code, a new military GPS signal 
designed to further improve anti-jamming and secure 
access to GPS signals for military users. JPALS 
program officials stated they contracted for a trade 
study to determine future M-code integration and 
implementation options. Program officials expect the 
study to be delivered in early 2019. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
stated that JPALS is part of a family of systems that 
provide capability to naval aviation and its partners. 
According to the program office, in fiscal year 2018 and 
early fiscal year 2019, JPALS successfully deployed on 
the amphibious ships LHD 1 and LHD 2, supporting 
F-35 operational deployments. The program stated that, 
in fiscal year 2018, it received approval to compress the 
JPALS production schedule from five to four lots, which 
it anticipated would save costs over the program lifetime 
and accelerate deployment. The program also stated 
that JPALS entered the production and deployment 
phase on March 25, 2019, which it said provides 
authority to award a low-rate initial production contract 
for 23 JPALS quantities. The program said that it 
expects to complete some integrated testing and an 
operational assessment in April 2019 in support of 
JPALS’s integrated operational test and evaluation 
phase. Additionally, the program stated that 
restructured and accelerated requirements drove 
changes to design drawings during JPALS 
development. 
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Lead Component: Navy 


Common Name: LCS Packages 



Source: U S. Navy. 


Littoral Combat Ship-Mission Modules (LCS Packages) 

The Navy’s LCS packages—weapons, helicopters, boats, and sensors 
launched and recovered from LCS seaframes—will provide mine 
countermeasure (MCM), surface warfare (SUW), and antisubmarine 
warfare (ASW) capabilities to the LCS seaframe. The Navy originally 
planned to swap mission packages among LCS, but has since decided 
that each LCS will be designated with a specific mission package. The 
Navy is currently delivering specific systems as they become available, 
rather than complete packages. We assessed the current capability of 
the delivered systems against the threshold requirements that define the 
baseline capabilities for the complete mission package._ 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 
Program office: Washington Navy Yard, 
DC 

Prime contractor: Northrop Grumman 


First full estimate 
(08/2007) 

Latest 

(09/2018) 

Percentage 

change 

Development 

NA 

$2,724.90 

NA 

Contract types: FFP/FPI (production) 

Procurement 

$3,649.49 

$3,771.82 

+3.4% 

Software development approach: 

Incremental 

Unit cost 

NA 

$133.43 

NA 

Next major milestone: Surface warfare 
package initial capability with surface-to- 

Acquisition cycle 
time (months) 

NA 

NA 

NA 

surface missile (second quarter, fiscal 

Total quantities 

64 

49 

-23.4% 


The Navy approved an acquisition program baseline for LCS Packages in September 2018. Total quantities 
comprise 5 development quantities and 44 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 




■ Funded to date 
MTo complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 
Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

Complete a system-level preliminary design review 


Product design is stable 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


(Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Lead Component: Navy 

LCS Packages Program 

Mine Countermeasure 

The Navy accepted systems that comprise about seven 
MCM packages prior to demonstrating that they meet 
threshold MCM requirements. It continues to fund 
numerous retrofits and configuration changes due to 
reliability and performance failures. Rather than 
complete packages, the Navy is now delivering MCM 
systems individually. Navy officials stated that they no 
longer plan to conduct testing to demonstrate that the 
packages meet MCM requirements because it believes 
that as long as each system meets its specific 
requirements, collectively it will meet overarching MCM 
requirements. The Navy has yet to demonstrate that 
meeting specific system requirements satisfies 
overarching MCM requirements. 

The Navy projects that it will not meet its requirement 
for Deep Volume Focused Mine Hunting, which involves 
the clearing of all moored mines. Under the current 
requirement, the package must clear a certain number 
of squared nautical miles each day. Officials said that 
the original requirement was optimistic and that the 
technology has not performed as needed to meet it, 
despite previous claims by the Navy that the systems 
were fully mature. The program office is currently 
working with stakeholders to change the requirement 
and align it with operator needs. Officials said they are 
seeking approval for an updated requirement in March 
2019. 

Following our 2018 assessment, the program delayed 
initial operational capability (IOC) for the MCM package 
from 2020 to 2022. The Navy plans to start fielding eight 
LCS with the MCM package between 2020 and 2022. 
These ships will deploy with only selected MCM 
systems that do not provide the package’s full 
capability. Officials stated that because of the variability 
within MCM missions, they can conduct some 
operations with the MCM systems that have already 
achieved IOC. Navy officials are also considering 
fielding these systems on other ships apart from LCS. 

To realize these plans, the Navy plans to procure 24 
MCM packages, but will use only 15 of them on LCS. 

Surface Warfare 

Through operational testing completed in fiscal year 
2015, the Navy met its incremental performance 
requirements for the SUW package and is now fielding 
it. To meet full performance requirements, the Navy is 
adding the Army’s Longbow Hellfire missile to the 
package, which the Navy has adapted and is currently 
testing for use within a maritime environment. According 
to officials, test ship availability issues delayed IOC for 
the missile from the second quarter of fiscal year 2018 
to the second quarter of fiscal year 2019. The Navy 
completed testing on one LCS variant (Freedom). The 


Common Name: LCS Packages 

Navy plans to test the missile on the other variant 
(Independence) in 2020. 

Antisubmarine Warfare 

Following our 2018 assessment, the program delayed 
IOC for the package by 1 year to the end of fiscal year 
2020 because of budget limitations, according to 
program officials. To meet this new date, the Navy 
accepted delivery of the Escort Mission Module test 
article, one of the package’s main systems, and plans to 
begin testing in the fourth quarter of fiscal year 2019 on 
board LCS 3. To achieve this, the Navy needs to make 
alterations on LCS 3 to outfit it with the ASW package. 
However, the program cannot begin the alterations until 
LCS 3 completes ongoing maintenance and subsequent 
sea trials, according to ship officials. The ASW 
alterations are scheduled to begin in April 2019, a 2- 
month delay from the previous start date. At this point, 
the Navy has little to no schedule margin remaining 
between when it expects to complete the LCS 3 
alterations and begin ASW package testing. Any further 
delays will jeopardize the Navy’s ability to meet its 
planned ASW package testing and IOC dates. 

Other Program Issues 

The Navy recently reduced its procurement quantities of 
LCS mission packages from 64 to 44, trimming the 
planned 24 SUW and 16 ASW packages to 10 each. 
This is in line with reductions to the total number of LCS 
from 52 to 32. Additionally, the Navy recently decided 
that it will semi-permanently install mission modules on 
LCS seaframes—an approach that it expects will add 
simplicity and stability to the program but will reduce the 
mission flexibility that the Navy previously intended the 
program to have. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. Officials said that the Navy is 
delivering LCS capability and deploying mission 
package systems that have attained IOC. They stated 
that the Navy successfully completed surface-to-surface 
missile module testing 2 months early, that the missile 
will achieve IOC as scheduled in the second quarter of 
fiscal year 2019, and that it will deploy on LCS 7 in the 
fourth quarter of fiscal year 2019. Officials also said that 
the Navy took delivery of the ASW mission package in 
November 2019, plans for it to embark on LCS 3, and to 
begin formal testing of it in fiscal year 2019 to achieve 
the scheduled fiscal year 2020 IOC. According to the 
program, the Navy has established IOC for three MCM 
mission package airborne systems. The program stated 
that it expects the full package’s IOC to occur in fiscal 
year 2022 due to funding limitations for the MCM 
unmanned surface vehicle. 
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Lead Component: Navy 


Common Name: MQ-4C Triton 



Source: MQ-4C Triton B-5 


MQ-4C Triton Unmanned Aircraft System (MQ-4C Triton) 

The Navy's MQ-4C Triton is an unmanned aircraft system, based on 
the design of the Air Force's RQ-4B Global Hawk air vehicle. It will be 
operated from five land-based sites worldwide and provide the Navy 
with persistent maritime intelligence, surveillance, and reconnaissance 
(ISR) data collection and dissemination capability. The Triton is integral 
to a family of maritime patrol and reconnaissance systems and part of 
the Navy's plan to recapitalize its airborne ISR assets by the end of the 
decade. 
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Program Essentials 

Milestone decision authority: Navy 
Program office: Patuxent River, MD 
Prime contractor: Northrop Grumman 

Contract types: Cost-sharing 
(development) 

FPI (low-rate initial production) 

FFP (low-rate initial production spares) 

Software development approach: 

Mixed 

Next major milestone: Baseline early 
capability ( July to September 2019) 


Program Performance (fiscal year 2019 dollars in millions) 


First full estimate 
(02/2009) 

Latest 

(10/2018) 

Percentage 

change 

Development 

$3,517.70 

$5,668.00 

+61.1% 

Procurement 

$10,439.20 

$9,760.90 

-6.5% 

Unit cost 

$205.50 

$225.24 

+9.6% 

Acquisition cycle 

92 

156 

+69.6% 

time (months) 




Total quantities 

70 

70 

0.0% 


Total quantities comprise 5 development quantities and 65 procurement quantities. Since last year’s report, one unit 
shifted from procurement to development as part of an agreement with the contractor to share cost growth. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

S715.21 _ ^ $4.952.82 


o 


Procurement 

$1.946.40 


$7.814.53 ^ ^ 


Procurement Quantities 


56 


cr 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
|To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Common Name: MQ-4C Triton 


Lead Component: Navy 

MQ-4C Triton Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The MQ-4C Triton program currently relies on no critical 
technologies and has stabilized its system design as it 
continues low-rate initial production. 

Program officials acknowledged that although the wing 
contractor continues to improve its manufacturing 
processes, production remains out of statistical control, 
which poses continued risk to the Triton’s production 
schedule, quality, and cost. For example, program 
officials said the contractor reduced defects by 68 
percent between the wings manufactured for the last 
system development test article—the sixth aircraft to be 
produced—and those manufactured for the tenth 
aircraft. In addition, the 10th and 11th sets of wings, 
which are the most recently produced, were each 
delivered 2 to 4 weeks ahead of schedule. Program 
officials emphasized that continued improvement will 
depend on the contractor’s ability to successfully 
implement corrective actions, such as updating 
engineering processes and establishing a dedicated 
team of mechanics to address specific quality issues. 

Program officials reported the start of full-rate 
production has been extended an additional 2 months, 
until July 2021. In addition, these problems have 
increased the program’s total number of design 
drawings by nearly 14 percent since receiving Milestone 
C approval in September 2016. The program plans to 
increase the quantity of aircraft acquired during low-rate 
initial production from 15 to 18 to maintain operational 
availiability requirements. However, this strategy also 
increases the quantity of aircraft that could require 
design changes due to testing discoveries as the Triton 
completes developmental testing and begins 
operational testing. 

Other Program Issues 

Program officials reported they potentially face further 
delays to the start of full-rate production after an 
aviation mishap—an unplanned event or series of 
events that results in damage to property or injury to 
personnel—occurred on September 12, 2018, during 
early operational testing. While program officials report 
that the Navy has completed an internal investigation 
that did not detect any system design deficiencies, the 
program paused flight test of all aircraft for 3 months out 
of an abundance of caution while the internal 
investigation was underway. The program also made 
updates to the training materials and operation 
procedures to address findings from their investigation. 
Program officials reported that the Triton returned to 
flight during the week of December 17, 2018, and early 
operational testing resumed on January 23, 2019. 

Program officials also told us that they have taken over 
configuration and development of the Triton’s Logistics 


Management System software from the contractor, as 
originally planned. This system, according to program 
officials, provides health and maintenance data on 
aircraft subsystems, including real-time notification of 
part failures as they occur during flight. While program 
officials stated the Navy is still designing and testing the 
software, the program plans for the software to be 
operational prior to initial operational capability in April 
2021. 

Program Office Comments 

We provided a draft assessment to the program office 
for review and comment. The program office provided 
technical comments, which we incorporated where 
appropriate. 
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Lead Component: Navy 


Common Name: MQ-25 Stingray 



Source: Jeff Hobrath, Chief, USN (Ret) President, Naval Trees, LLC. 
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MQ-25 Unmanned Aircraft System (MQ-25 Stingray) 

The Navy’s MQ-25 is a catapult-launched unmanned aircraft system 
that will operate from aircraft carriers. The Navy expects MQ-25 to 
provide a refueling capability for the carrier air wing and the 
intelligence, surveillance, and reconnaissance capabilities needed to 
identify and report on surface targets, such as ships. The system is 
made up of an aircraft segment, a control station segment, and a 
carrier modification segment. The aircraft development represents 
about 90 percent of the Navy’s planned investment in the MQ-25 
system as a whole over the next 5 years. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 
Program office: Patuxent River, MD 


First full estimate 
(08/2018) 

Latest 

(08/2018) 

Percentage 

change 

Prime contractor: Boeing 

Contract type: FPI (development) 

Development 

$3,548.60 

$3,548.60 

0.00% 

Software development approach: 

Procurement 

$8,915.00 

$8,915.00 

0.00% 

Mixed 

Unit cost 

$168.85 

$168.85 

0.00% 

Next major milestone: Design review 
(December 2019) 

Acquisition cycle 
time (months) 

72 

72 

0.00% 


Total quantities 

76 

76 

0.00% 


The Navy approved an acquisition program baseline for the MQ-25 Stingray program in August 2018. Total 
quantities comprise 4 development quantities and 72 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 

$3.383.50 __ $ 165.0 6 


0 


Procurement 



Procurement Quantities 

■ Funded to date 

■ To complete 


Attainment of Product Knowledge 

As of January 2019 



Status at 

Current Status 

Resources and requirements match 

Development 

Start 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• 

• 



• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

O 

O 

• Complete a system-level preliminary design review 

O 

O 

Product design is stable 

Design Review 


• Release at least 90 percent of design drawings 

NA 

NA 

• Test a system-level integrated prototype 

NA 

NA 

Manufacturing processes are mature 

Production Start 


• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

NA 

NA 

• Demonstrate critical processes on a pilot production line 

NA 

NA 

• Test a production-representative prototype in its intended 
environment 

NA 

NA 


iKnowledge attained 


B 


Knowledge not attained, 


(information not available, 




Not applicable 
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Lead Component: Navy 

MQ-25 Stingray Program 

Technology Maturity and Design Stability 

The Navy identified no critical technologies for MQ-25. 
The Navy approved the MQ-25 technology readiness 
assessment in June 2018 as part of the program’s 
development start decision. The MQ-25 program has 
other critical technologies not resident in the MQ-25 
aircraft. The Navy is developing those two critical 
technologies, which are currently approaching maturity, 
under its Joint Precision Approach and Landing System 
program. Our Attainment of Product Knowledge table 
accounts for these technologies. The program has 
initiated system design activities ahead of a planned 
review in December 2019. The Navy did not hold a 
preliminary design review for MQ-25 prior to the start of 
development—an approach inconsistent with best 
practices. Instead, the program has relied on 
preliminary design knowledge attained under the 
previous Unmanned Carrier-Launched Airborne 
Surveillance and Strike (UCLASS) program to inform 
MQ-25 development. Nonetheless, the Navy has set 
different performance requirements for MQ-25 as 
compared to UCLASS, which necessitates design 
differences between the two aircraft. 

In an effort to accelerate acquisition and fielding, the 
Navy designated MQ-25 a Maritime Accelerated 
Acquisition program. This designation is intended to 
enable the MQ-25 program to deliver capabilities more 
rapidly than it could under the traditional DOD 
acquisition process. The Navy emphasized schedule 
during the source selection process, and, according to 
program officials, Boeing presented a schedule that 
would allow the program to achieve initial operational 
capability 2 years earlier than originally planned. 
Program officials stated that, as part of this acquisition 
approach, they plan to hold periodic design reviews as 
MQ-25 subsystems are completed. The reviews will 
conclude with a final system design review in December 
2019, at which point the program expects to have 
completed its final design. 

Production Readiness 

The Navy plans to authorize MQ-25’s entry into low-rate 
initial production in October 2023. Boeing plans to use a 
commercially demonstrated approach to manufacture 
the aircraft in modules, which program officials say will 
be faster and more efficient than a typical assembly 
line. In this approach, suppliers will use detailed models 
to produce the various aircraft modules, including pre¬ 
drilling holes. The suppliers will then send the 
completed modules to Boeing to assemble. 

Under MQ-25’s approach, the program could find that 
critical manufacturing processes reside with the 
suppliers responsible for module fabrication. According 
to program officials, Boeing intends to conduct 
manufacturing readiness level assessments when 


Common Name: MQ-25 Stingray 

building the aircraft. Until the program develops the 
knowledge that its critical manufacturing processes are 
in statistical control, there is increased risk that the 
design may not be producible at the program’s cost, 
schedule, and quality targets. Best practices call for 
programs to attain this knowledge prior to entering 
production. 

Other Program Issues 

Officials reported that in August 2018, the Navy 
awarded a fixed price incentive development contract 
with a ceiling price of $805 million to Boeing, which 
includes four MQ-25 aircraft. Prior to the award, the 
Navy estimated the total development cost at $3.8 
billion, and the Office of the Secretary of Defense 
(OSD) estimated its cost at $5.1 billion. Based on the 
disparity between the development contract’s ceiling 
price and the DOD cost estimates, OSD cost estimators 
emphasized the importance of avoiding requirements 
changes that might lead to the renegotiation of the 
development contract’s terms, including the ceiling 
price. 

Program officials stated the development contract 
includes options for up to three additional test aircraft, 
but does not include options for any of the 72 planned 
production aircraft. Officials stated that while they do not 
plan to compete the production contract, Boeing will 
provide cost, technical, and programmatic data during 
the development phase, which is intended to help 
inform the Navy’s negotiation of a fair and reasonable 
price under the production contract. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. 


Page 118 


GAO-19-336SP Weapon Systems Annual Assessment 




Lead Component: Navy 


Common Name: NGJ Mid-Band 



Source: Raytheon Corp. 


Next Generation Jammer Mid-Band (NGJ Mid-Band) 

The Navy’s Next Generation Jammer (NGJ) is an external jamming 
pod system that will be integrated on EA-18G Growler aircraft. It will 
augment, then replace, the ALQ-99 jamming system and provide 
enhanced airborne electronic attack capabilities to disrupt adversaries’ 
use of the electromagnetic spectrum for radar detection, among other 
purposes. The Navy plans to field the system that jams mid-band radio 
frequencies in 2022. The Navy has a separate program for low-band 
frequencies and will roll out a high-band frequencies program at a later 
date. We assessed the Mid-Band program. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 
Program office: Patuxent River, MD 


First full estimate 
(04/2016) 

Latest 

(06/2018) 

Percentage 

change 

Prime contractors: Raytheon; Boeing 
Contract type: CPIF (development— 

Development 

$3,629.80 

$4,064.20 

+12.0% 

Raytheon) (development and 

Procurement 

$4,206.20 

$4,062.3 

-3.4% 

integration—Boeing) 

Unit cost 

$58.10 

$60.25 

+3.7% 

Software development approach: 

Mixed 

Acquisition cycle 

98 

110 

+ 12.2% 

Next major milestone: Low-rate initial 
production (August 2020) 

time (months) 

Total quantities 

135 

135 

0.00% 


Total quantities comprise 7 development quantities and 128 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$1.528.90 ^ $2.535.31 




Procurement 


$4.062.32 


O 


Procurement Quantities 


128 


O 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
|To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Lead Component: Navy 

NGJ Mid-Band Program 


Common Name: NGJ Mid-Band 


Technology Maturity and Design Stability 

Nearly 3 years after development start, the NGJ Mid- 
Band program’s seven critical technologies continue to 
approach maturity—an approach inconsistent with best 
practices. The program also reports that it has released 
all of its planned design drawings following a redesign 
of the jamming pod to address structural issues. 
However, until the program fully matures its critical 
technologies—by demonstrating each in a final form, fit, 
and function within a realistic environment—existing 
design drawings face risk of change. Further, the 
program has yet to test a system-level integrated 
prototype, which best practices hold as a key criterion 
for achieving design stability. 

The program entered system development in April 2016 
with its critical technologies approaching maturity. 

These technologies include two separate arrays—each 
with different transmit/receive modules, circulators, and 
apertures—as well as a power generation system. The 
program plans to have its critical technologies fully 
mature, integrated, and in testing in January 2020. 

The NGJ Mid-Band program discovered design 
deficiencies with the jamming pod structure at its April 
2017 critical design review, which caused a 1-year 
schedule delay and contributed to an over $400 million 
increase in the program’s development cost. The 
program identified several other deficiencies in 
modeling, assumptions, and methodologies regarding 
the structural analysis and design of the pod structure, 
which required redesign. According to the program 
office, the pod redesign did not affect the program’s 
critical technologies, subsystems, or software. 

As of September 2018, the contractor had released 100 
percent of the design drawings. Program officials said 
that they considered the design of the pod structure to 
be stable, although the contractor continues to make 
minor changes to it. In addition, the program office has 
yet to test a system-level integrated prototype of the 
jamming pod, which runs counter to the GAO-identified 
best practice for demonstrating design stability. The 
program office plans to begin testing the redesigned 
pod on an EA-18G in January 2020. 

Production Readiness 

The program office plans to demonstrate its critical 
manufacturing processes prior to the start of production 
in August 2020, which would be an approach consistent 
with best practices. However, the program office does 
not plan to test a production-representative prototype or 
complete system-level developmental testing (which 
includes demonstrating the full functionality of the 
system) until 7 and 15 months, respectively, after 
production starts. DOD policy allows some concurrency 
between developmental testing and initial production, 
but we have previously found that starting production 


before demonstrating that a system will work as 
intended increases the risk of deficiencies that require 
substantial and costly design changes. Program officials 
told us that they plan to mitigate the risk associated with 
the concurrency between developmental testing and 
initial production to what they consider to be an 
acceptable level by gathering extensive data about pod 
performance in specialized ground test chambers and 
through flight testing engineering development models. 
Further, according to program officials, the delay 
caused by the pod redesign will allow the program to 
conduct additional ground testing before committing to 
production. 

In addition, the delay associated with the pod redesign 
introduced a potential production gap in between the 
last engineering developmental model and the first 
production system. The program added three additional 
system demonstration test articles to mitigate this gap 
and verify that the system can meet all its performance 
requirements prior to the start of operational testing in 
January 2022. 

Other Program Issues 

The Under Secretary of Defense for Acquisition, 
Technology, and Logistics approved NGJ Mid-Band as 
the first program in the “Skunk Works” pilot in 
September 2015, which aims to streamline processes in 
order to deliver capabilities on time and within budget. 
NGJ Mid-Band officials said that the pilot helped reduce 
the length of time spent on the decision making process 
by providing officials direct access to management and 
allowing them to focus on execution. Nonetheless, the 
program has still exceeded original estimates for 
development cost, unit cost, and schedule. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. According to program officials, as of 
January 2019, all subsystems that contain critical 
technologies have been manufactured and assembled 
and are undergoing subsystem-level tests. Program 
officials reported that this progress has reduced 
program risk to an acceptable level for system 
development. Additionally, program officials told us that 
they plan to test a system-level integrated prototype of 
the pod on an EA-18G in a sound- and electromagnetic- 
absorbent chamber beginning in the fourth quarter of 
fiscal year 2019. 
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Lead Component: Navy 


Common Name: OASuW Inc 1 



Offensive Anti-Surface Warfare Increment 1 (OASuW Inc 

1) 

The Navy’s OASuW Inc 1 program plans to develop an air-launched, 
long-range, anti-surface warfare missile to address an urgent 
operational need. The program is using an accelerated acquisition 
approach and has leveraged previous technology demonstration 
efforts. It fielded an early operational capability on Air Force B-1 
bombers in 2018 and plans to do so on Navy F/A-18 aircraft in 2019. 
DOD also plans to develop an additional capability with Increment 2 to 
address future threats. We assessed Increment 1. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 
Program office: Patuxent River, MD 


First full estimate 
(03/2016) 

Latest 

(02/2019) 

Percentage 

change 

Prime contractor: Lockheed Martin 

Contract type: CPIF (development) 

Development 

$1,263.30 

$1,466.80 

+16.1% 

FPIF (low-rate initial production) 

Procurement 

$314.30 

$1,319.30 

+319.8% 

Software development approach: 

Unit cost 

$12.72 

$7.14 

-43.8% 

Incremental approach 

Acquisition cycle 

42 

42 

0.00% 

Next major milestone: Early operational 
capability with F/A-18 (September 2019) 

time (months) 

Total quantities 

124 

390 

+214.5% 


We calculated acquisition cycle time using the program’s early operational capability with F/A-18 as its initial 
operational capability. Total quantities comprise 16 development quantities and 374 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$275.46 _ ^ $1.191.37 


o 


Procurement 


$1.010.32 


Procurement Quantities 
84 



Attainment of Product Knowledge 

As of January 2019 


Funded to date 
|To complete 


Status at 


Current Status 


Resources and requirements match 


Development 

Start 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Common Name: OASuW Inc 1 


Lead Component: Navy 


OASuW Inc 1 Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The OASuW Inc 1 program has mature technologies, a 
stable design, and production practices approaching 
maturity. The Navy chose to move forward earlier in the 
program without the technology, design, or production 
knowledge recommended by best practices. In 2016, 
the program office acknowledged the risk its strategy 
posed, but pointed to an urgent maritime warfighting 
need as well as steps taken to manage the risk, such as 
leveraging the Air Force’s Joint Air-to-Surface Standoff 
Missile-Extended Range (JASSM-ER) airframe and 
production facilities. 

According to the program office, the Navy assessed 
OASuW’s manufacturing processes as mature prior to 
production. However, the program’s last formal 
assessment in March 2017 indicated its processes were 
only approaching maturity. The program’s top priority 
has been to produce missiles to field an early 
operational capability on the B-1 and F/A-18 aircraft. 

The program delivered the last missiles needed to 
achieve that capability on the B-1 later than planned in 
2018. Consequently, the B-1 achieved early operational 
capability in December 2018—3 months later than the 
program’s goal, but 9 months earlier than required. 

The program is now planning future changes to the 
missile to improve performance, counter evolving 
threats, and address parts obsolesence. Some changes 
will involve hardware updates that are currently in 
development; others will leverage upgrades from the 
JASSM-ER program. For example, according to 
program officials, they plan to incorporate an enhanced 
weapon data link. They expect to begin incorporating 
this data link into missiles starting with the fourth 
OASuW production lot, which the contractor is 
scheduled to begin in the third quarter of fiscal year 
2020. The program office does not plan to upgrade 
already fielded missiles with this link. 

Other Program Issues 

The OASuW program relies on a combination of missile 
tests and modeling and simulation to demonstrate and 
characterize the system’s performance. One key 
modeling and simulation facility, however, was not yet 
operational as of November 2018. According to 
program officials, the program uses modeling and 
simulation in part because replicating the operational 
environment is challenging. The program completed six 
flight tests in 2017 and 2018, among other activities to 
demonstrate missile performance. Navy officials stated 
that, because it did not have all the anticipated 
modelling and simulation capabilities available before 
fielding the OASuW early operational capability on the 
B-1, the program did not collect as much information as 
planned about the weapon’s performance in a wide 
range of conditions. Officials expect the new modeling 


and simulation facility will be available to support testing 
for the F/A-18 early operational capability. 

Since the program’s 2016 development start, DOD has 
significantly increased OASuW Inc 1 quantities due to 
Increment 2 delays and evolving threats. Most recently, 
in February 2019, DOD increased planned procurement 
quantities from 110 to 374. According to program 
officials, the increases will provide sufficient quantities 
for mission needs until a future capability becomes 
available. Officials stated the Navy is examining options 
for addressing future threats and Increment 2 by 
building on its Next Generation Land Attack Weapon 
Analysis of Alternatives. The Navy plans to complete its 
OASuW-related analysis in 2019. The program office 
expects OASuW Increment 2 to achieve initial 
operational capability as early as fiscal year 2028. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that 
OASuW Inc 1 is an accelerated acquisition program 
intended to fill an urgent, unmet maritime warfighting 
requirement where schedule is a priority. The program 
office said that a small, integrated team of government 
and industry experts manages the program, including 
the risks it faces related to concurrent weapon system 
development, test, and production. It also stated that 
monthly executive steering boards chaired by the 
Assistant Secretary of the Navy for Research, 
Development, and Acquisition provide oversight. The 
program office reiterated its position that the OASuW’s 
production processes are mature. It also stated that 
OaSuW successfully completed both a production 
readiness review and physical configuration audit, and 
the contractor has delivered 10 missiles to date. 
Additionally, the program office said that it met the early 
operational capability date for fielding OASuW on the B- 
1 bomber aircraft, and the program is ahead of 
schedule for fielding OASuW Inc 1 on F/A-18 fighter 
aircraft no later than the fourth quarter of fiscal year 
2019. According to the program office, the Navy intends 
to procure OaSuW quantities based on the available 
funding, which may cause purchased quantities to differ 
from current projections. 
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Lead Component: Navy 


Common Name: SSBN 826 



Source: General Dynamics Electric Boat. 


SSBN 826 Columbia Class Ballistic Missile Submarine 
(SSBN 826) 

The Navy's Columbia class (SSBN 826) will replace its current fleet of 
Ohio class ballistic missile submarines, which the Navy plans to retire 
starting in 2027. The submarine will serve as a sea-based, strategic 
nuclear deterrent that will remain in service through 2080. According to 
the Navy’s current acquisition plan, the lead ship will make its first 
patrol in October 2030. 
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Program Essentials 


Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Under 
Secretary of Defense for Acquisition and 
Sustainment 

Program office: Washington Navy Yard, 
DC 

Prime contractor: General Dynamics 
Electric Boat 

Contract type: Combination of CPIF and 
CPFF (design and development) 

Software development approach: 

Mixed 

Next major milestone: Design review 
(April 2020) 


First full estimate 
(01/2017) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$13,068.90 

$13,103.00 

+0.3% 

Procurement 

$90,335.30 

$89,932.20 

-0.4% 

Unit cost 

$8,629.70 

$8,599.85 

-0.3% 

Acquisition cycle 

231 

237 

+2.6% 

time (months) 




Total quantities 

12 

12 

0.00% 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 

$3.616.34 ^ $9.486.68 




Procurement 



Procurement Quantities 


Attainment of Product Knowledge 

As of January 2019 


Status at 

Current Status 

Detail Design 

Resources and requirements match Contract Award 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

O 

O 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

O 

O 

• Complete a system-level preliminary design review 



• 

• 

Product design is stable Fabrication Start 


• Complete basic and functional design to include 100 
percent of 3D product modeling 

NA 

NA 


[Knowledge attained, Q Knowledge not attained, ■information not available, Not applicable 


Jr 


O 

■ Funded to date 
■To complete 
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Lead Component: Navy 

SSBN 826 Program 

Technology Maturity 

The Columbia class program identified two critical 
technologies—a carbon dioxide removal system and the 
stern area system, the details of which are classified. 
The program expects the carbon dioxide removal 
system to reach full maturity in late 2019, while the 
stern area system is still immature. 

In December 2017, we reported that several Columbia 
class technologies that met GAO’s definition of a critical 
technology element were not identified by the Navy as 
critical technologies. Specifically, the Navy did not 
follow best practices for assessing critical technologies. 
When we applied these best practices, we identified 
four additional critical technologies that the Navy 
excluded. These include the integrated power system, 
the propulsor/coordinated stern, the common missile 
compartment (CMC), and the nuclear reactor. Of these, 
only the nuclear reactor is fully mature as of late 2018. 

The Navy expects the CMC to reach full maturity in 
2019. However, officials reported that in July 2018 the 
shipbuilder identified significant weld defects in CMC 
missile tubes from one of three suppliers after the 
supplier had already delivered seven tubes to the 
shipyard and installation work had begun, resulting in 
rework. Officials further report that the shipbuilder found 
defects affected five additional tubes. Program officials 
attributed these defects to inexperienced welders and 
weld inspectors. The Navy estimates that, as of January 
2019, the CMC consumed 52 percent of its schedule 
margin. Should the Navy discover additional CMC 
deficiencies, the planned construction sequence for the 
lead submarine will be jeopardized. 

Further, manufacturing defects have delayed delivery of 
the integrated power system’s (IPS) first production- 
representative motor. The Navy plans to recover the 
motor’s schedule margin by testing it while the supplier 
updates the motor’s production design. Consequently, 
any new deficiencies discovered in testing may require 
the supplier to modify its design, which could delay the 
lead ship’s IPS motor production schedule. 

Design Stability 

The program office plans to complete the basic and 
functional design prior to the lead submarine’s 
scheduled construction start, in October 2020. 

However, Navy officials report the shipbuilder has 
already begun building sections of the submarine, with 
95 percent of the basic and functional design 
complete—a level slightly below best practices. Further, 
the Navy has determined that the shipbuilder needs to 
complete 83 percent of the detail design—the most 
complex design phases down to the lowest level of the 
submarine—by October 2020 to meet its cost and 
schedule goals. Currently, the shipbuilder is behind 
schedule because it has yet not achieved planned 


Common Name: SSBN 826 

efficiencies with new design software. The shipbuilder 
increased its design staff by 18 percent in an effort to 
reach the design goal on schedule. However, the 
program’s plan for achieving design stability is premised 
on assumptions about the final form, fit, and function of 
critical technologies—and how those technologies will 
perform in a realistic environment—that the program 
has yet to demonstrate. 

Production Readiness 

By beginning to build sections of the submarine starting 
in December 2018, the Navy believes that the builder 
can achieve an aggressive 84-month construction 
schedule. However, this is 2 years prior to the planned 
request for fiscal year 2021 authorization to start 
construction of the lead ship. 

Other Program Issues 

In a April 2019 report, we made several 
recommendations to improve the program’s cost 
estimate. Specifically, we found that the program’s $115 
billion procurement cost estimate is not reliable because 
its estimate is based on overly optimistic assumptions 
about the labor hours needed to construct Columbia 
class submarines and did not include any cost margin in 
case these assumptions are not met. While the Navy 
analyzed program cost risks, it did not include enough 
margin in its estimate for likely cost growth. The Navy 
plans to update the cost estimate for the lead ship, but it 
may not complete this update in time for its fiscal year 
2021 budget request, which will seek authorization and 
funding for lead submarine construction. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that it 
intends to provide needed capabilities on schedule and 
at an affordable price by committing to stable 
requirements, achieving high design maturity at the start 
of construction for the lead submarine, improving 
manufacturing and construction readiness, and 
aggressively working to reduce costs. It also said it 
plans to complete 83 percent of the design by 
construction start—more than other recent submarine 
programs. The program also stated that it plans to 
update its cost estimate in 2019 to inform lead 
submarine funding. The program noted that the Navy 
recognizes its supplier base remains a high risk to 
construction readiness and continues to devote 
increased oversight on manufacturing issues and 
readiness assessments. The program said it continues 
to comply with all Navy, Department of Defense, and 
statutory requirements for managing critical 
technologies. 
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Lead Component: Navy 


Common Name: SSC 



Ship to Shore Connector Amphibious Craft (SSC) 

The Navy's SSC is an air-cushioned landing craft intended to transport 
personnel, weapon systems, equipment, and cargo from amphibious 
vessels to shore. SSC is the replacement for the Landing Craft, Air 
Cushion, which is approaching the end of its service life. The SSC is 
designed to deploy in and from Navy amphibious ships that have well 
decks, such as the LPD 17 class, and will support assault and non¬ 
assault operations. 


Source: Textron Marine and Land Systems. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Navy 


First full estimate 

Latest 

Percentage 

Program office: Washington, DC 


(07/2012) 

(06/2018) 

change 

Prime contractor: Textron Inc. 

Contract type: FPI (detail design and 

Development 

$623.20 

$586.20 

-5.9% 

construction) 

Procurement 

$3,782.20 

$4,122.30 

+9.0% 

CPFF (long lead materials and early 
production) 

Unit cost 

$60.63 

$64.72 

+6.7% 

Software development approach: 

Modified waterfall 

Acquisition cycle 
time (months) 

135 

135 

0.0% 

Next major milestone: Start operational 

Total quantities 

73 

73 

0.00% 


testing (summer 2020) 


Total quantities comprise 1 development quantity and 72 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$ 3.38 _ , $ 582.82 


o 


Procurement 


$ 3 . 153.90 ^ - $9 ^ 44 

c 


54 


Procurement Quantities 

XT 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
|To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Lead Component: Navy 

SSC Program 


Common Name: SSC 


Technology Maturity and Design Stability 

SSC’s one critical technology, the fire supression 
system, is mature. The SSC’s design was not stable at 
its design review in 2014, but the program now reports 
that it has achieved stability. However, software 
development—a modified waterfall approach, which 
includes iterative testing—is not yet finished. 
Additionally, according to the program office, problems 
with two components have required design changes 
and delayed the start of operational testing. 

The program office stated the gearbox, which had 
previous problems with premature wear, showed gear 
slippage during testing. The program is now working on 
separate engineering solutions for the propulsion and lift 
elements. This is the third iteration of design changes 
related to problems with the gearbox. The program 
expects that once the revised design is finalized and 
manufactured, the contractor will have to retrofit the 
craft already in production. 

In October 2018, during pre-delivery testing, Craft 
100—the test and training prototype—lost propulsion 
and drifted into a bridge, causing damage to one side of 
the craft. Program officials said the Navy found the root 
cause was overheating in the propulsion power control 
module. The prime contractor since developed a heat 
exchanger and, in November 2018, demonstrated it 
could significantly lower the module’s internal 
temperature. The program office said it plans to apply 
this solution to all craft currently in production. 

The program office will have the opportunity to 
demonstrate that the SSC design meets performance 
requirements through testing when the contractor 
delivers Craft 100, expected in late fiscal year 2019. 

The Navy and contractor will test Craft 100 while the 
contractor is producing the eight other SSC craft. This 
concurrency increases the risk that the program will 
discover deficiencies that could require costly design 
changes and modifications to units in production. 

Production Readiness 

The program entered low-rate initial production in May 
2015 after demonstrating that all materials, manpower, 
tooling, and facilities were proven and available to meet 
the production schedule, as recommended by DOD 
guidance. However, best practices indicate programs 
should also demonstrate that manufacturing processes 
are in statistical control prior to production start, which 
the program has not done. Critical processes should be 
repeatable, sustainable, and consistent in producing 
parts within the quality standards, which provides 
confidence that the product can be produced within 
cost, schedule, and quality targets. 

The aforementioned technical problems have caused 
delays to the SSC production schedule. Program 


officials reported nine SSC craft are currently under 
contract, and they are uncertain about the production 
schedule because solutions to the technical problems 
are pending. However, they expect delivery of Craft 100 
in July 2019, a 15-month delay compared to last year’s 
assessment. The program expects delivery of the 
second craft several months after Craft 100. 

Other Program Issues 

The program office will not verify in realistic operational 
conditions that it has fully addressed all known 
deficiencies until operational testing of Craft 100. 
However, SSC’s operational testing—which cannot take 
place until the Navy takes delivery of Craft 100—will not 
begin until summer 2020. The program plans to 
conclude testing about 2 months before it declares 
initial capability in August 2020, when the first SSC craft 
are scheduled to deploy. Should the Navy discover 
deficiencies during operational testing, it may face a 
choice to delay initial capability or deliver SSC craft that 
are operationally ineffective or unsuitable, as it has 
done in other programs. 

The program office said it has yet to award its follow-on 
contract for the 15 additional craft funded in fiscal years 
2017, 2018, and 2019. The program delayed 
negotiations because of prior funding uncertainty, as 
well as the effort to address the gearbox problems. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that since 
our assessment, the program has delayed operational 
testing by 12 months, initial capability by 12 months, 
and full-rate production by 13 months from the dates 
shown in our program timeline. It said that the SSC 
program has made notable progress, but testing of Craft 
100 continues to pose challenges, and electrical system 
stability and command, control, communications, 
computers and navigation integration challenges must 
be resolved before it can complete pre-delivery testing. 
The program said that Craft 100 is scheduled to 
complete on-water, pre-delivery testing and deliver in 
summer 2019, with subsequent craft delivering in late 
2019. The program also stated that the Navy’s 2020 
budget request has reduced fiscal year 2020 craft 
quantities from eight craft to zero. Additionally, it said 
that, although one test and training craft and 72 
operational craft remain the requirement, the above- 
mentioned change in requested funding shifts SSC's full 
capability to fiscal year 2032, 3 years later than 
previously planned. The program stated that Textron 
has indicated in the past that eight craft per year are 
necessary to achieve economies of scale, but with zero 
craft allotted in fiscal year 2020, the program foresees 
increased risk for growth in acquisition cost, lifecycle 
cost, and industrial base instability. 
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Lead Component: Navy 


Common Name: T-AO 205 



Source: General Dynamics NASSCO 


John Lewis Class Fleet Replenishment Oiler (T-AO 205) 

The John Lewis Class Fleet Replenishment Oiler (T-AO 205) will 
replace the Navy’s 15 existing Henry J. Kaiser Class Fleet Oilers 
(T-AO 187), which are nearing the end of their service lives. The 
primary mission of the oiler is to replenish bulk petroleum products, dry 
stores and packaged cargo, fleet freight, mail, and personnel to other 
vessels at sea. The Navy plans to procure a total of 20 ships. 
Construction of the lead ship began in September 2018. The Navy 
plans to proceed at a rate of one to two ships per year until 2033. 
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Program Essentials 

Milestone decision authority: Navy 
Program office: Washington Navy Yard, 
DC 

Prime contractor: General Dynamics 
National Steel and Shipbuilding Company 
(NASSCO) 

Contract type: FPI (detail design and 
construction) 

Software development approach: NA 

Next major milestone: Lead ship 
delivery (November 2020) 


Program Performance (fiscal year 2019 dollars in millions) 


F i rst f u 11 est i mate 
(09/2017) 

Latest 

(12/2018) 

Percentage 

change 

Development 

$71.00 

$70.80 

-0.3% 

Procurement 

$8,907.00 

$10,735.90 

+20.5% 

Unit cost 

$528.12 

$540.34 

+2.3% 

Acquisition cycle 

46 

52 

+ 13.0% 

time (months) 




Total quantities 

17 

20 

+ 17.6% 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 



Procurement Quantities 



Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Complete basic and functional design to include 100 
percent of 3D product modeling 


Detail Design 
Contract Award 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 



^Funded to date 
■To complete 
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Lead Component: Navy 

T-AO 205 Program 

Technology Maturity and Design Stability 

The Navy has matured all Lewis class critical 
technologies and stabilized the ships’ design. In 2014, 
the Navy identified three critical technologies for the 
Lewis class, all of which involved a new system for 
transferring cargo at sea. Prior to initiating detail design 
activities in June 2016, the Navy completed prototype 
tests of the critical technologies and found that they 
were fully mature—an approach consistent with 
shipbuilding best practices. In 2017, the Navy removed 
one critical technology—the Heavy e-STREAM cargo 
delivery system—from the Lewis class design. The 
Navy had intended to use this system to deliver F-35 
Lightning II power modules. The Navy subsequently 
decided to deliver these by air, which precluded any 
need for the Heavy system. 

Lead ship construction began in September 2018 with 
95 percent of the ship’s total design effort complete. 
Program officials stated that this figure meant that 100 
percent of the ship’s basic and functional design were 
by then complete—an approach consistent with best 
practices. Throughout detail design and now into 
construction, the Navy has not changed the Lewis class 
program’s performance requirements. The Navy also 
leveraged commercial vessel designs to minimize 
design and construction risks. The Lewis class features 
a modern double-hull construction, an environmental- 
based design standard for commercial tankers, to 
ensure the ships can dock at ports-of-call. This design 
was included in the final three Kaiser class oilers. 

Production Readiness 

The program office has largely kept to its construction 
schedule to date for the first ship, but a flooding incident 
at a NASSCO graving dock in July 2018 has affected 
the delivery of future ships. The program office stated 
that this incident has not affected current ship 
fabrication activities. However, the dock’s unavailability 
while repairs are planned and implemented has 
disrupted the contractor’s schedule for future ships. 
According to the program office, the incident has 
resulted in some delays to certain delivery dates for 
ships two through six. 

Other Program Issues 

As part of the Navy’s plan to expand the fleet, the Navy 
concluded that it would need an additional three Lewis 
class ships. The Navy’s budget request for fiscal year 
2019 increased its planned one-ship-per-year buy to 
two for fiscal years 2019, 2021, and 2023. The 
Congress provided appropriations for the additional 
fiscal year 2019 ship in support of the Navy’s request. 
To account for the additional ships in fiscal years 2019 
and 2021, the Navy plans to add two more ships to the 
low-rate initial production phase. Subsequently, 


Common Name: T-AO 205 

program officials stated that they plan to compete a new 
contract for the remaining 12 ships using the 
construction knowledge gained from efforts under the 
existing contract. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that it 
continues to follow GAO shipbuilding best practices and 
has leveraged commercial vessel design practices to 
minimize risk. The program office also stated that it is 
currently revising its acquisition baseline to reflect the 
update in total quantities to 20 ships. In addition, the 
program office noted that, in fiscal year 2019, it fully 
funded the third and fourth ships and funded advance 
procurement for the fifth ship. 
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Lead Component: Navy 


Common Name: VH-92A 



Source: USMC. 


VH-92A Presidential Helicopter Replacement Program 
(VH-92A) 

The Navy's VH-92A program provides new helicopters for safe, 
reliable, and timely transportation in support of the presidential airlift 
mission. It will replace the current Marine Corps fleet of VH-3D and 
VH-60N aircraft and supersedes the VH-71 program, which DOD 
canceled due to cost growth, schedule delays, and performance 
shortfalls. The VH-92A is expected to provide improved performance, 
survivability, and state-of-the-art communications capabilities, while 
offering increased passenger capacity. 
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Program Essentials 

Milestone decision authority Navy 
Program office: Patuxent River, MD 
Prime contractor: Sikorsky Aircraft 
Corporation, a Lockheed Martin 
Company 

Contract type: FPIF (development) 
FFP (production; planned) 

Software development approach: 

Mixed 

Next major milestone: Low-rate initial 
production (June 2019) 


Program Performance (fiscal year 2019 dollars in millions) 


First full estimate 
(04/2014) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$2,802.00 

$2,695.20 

-3.8% 

Procurement 

$2,197.20 

$2,140.00 

-2.6% 

Unit cost 

$217.35 

$210.23 

-3.3% 

Acquisition cycle 

75 

78 

+4.0% 

time (months) 




Total quantities 

23 

23 

0.00% 


Total quantities comprise 6 development quantities and 17 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$718.75 _ ^ $1.976.44 


o 


Procurement 
$2.139.99 


o 


Procurement Quantities 


171 


O 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
|To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Common Name: VH-92A 


Lead Component: Navy 

VH-92A Program 

Technology Maturity and Design Stability 

The VH-92A program has released all of its planned 
design drawings for this configuration that relies on no 
critical technologies. The program has a government- 
designed mission communications system (MCS), 
which consists of both government and commercial 
hardware and software. The program expects this 
system to provide the VH-92A passengers, pilot, and 
crew with the ability to conduct simultaneous short- and 
long-range secure and non-secure voice and data 
communications. The MCS is not in use in any other 
aircraft. The Navy plans to test MCS version 2.0 with 
the first two VH-92A developmental aircraft during an 
operational assessment scheduled to start in March 

2019. MCS version 2.0 has limitations connecting to 
secure networks, following a change to the network 
security required protocols. For the operational 
assessment, the Navy intends to connect to existing 
networks that do not use the new security protocols. 
This will allow the operational assessment to proceed, 
but will limit the scope of testing. The program is 
working on the problem, which the Navy anticipates will 
be resolved by December 2019, about 3 months prior to 
the start of operational testing scheduled for March 

2020. Once resolved, the Navy plans to upgrade the 
production representative aircraft with MCS version 3.0 
retrofit kits and use this version for operational testing. 

Based on new information provided by the program 
office, we have updated our attainment of product 
knowledge table to show that the program met the best 
practices metric of at least 90 percent of design 
drawings released to manufacturing at the time of its 
2016 critical design review. However, the program did 
not test a system-level integrated prototype at the time 
of its critical design review, which is required by best 
practices to confirm design stability. 

The program currently identifies two VH-92A 
performance attributes as high risks to the aircraft’s 
development. Specifically, the aircraft has yet to 
demonstrate performance requirements related to (1) 
the propulsion system and (2) landing zone suitability, 
which includes a requirement to land on the White 
House south lawn without causing damage to the lawn. 
Additionally, the program plans to implement design 
changes outside the baseline program that will allow for 
improved visibility from one of the aircraft’s doors. The 
aforementioned development activities are expected to 
be completed after the June 2019 production start. This 
approach introduces the potential for discovery of 
design deficiencies in testing at the same time that the 
contractor is manufacturing aircraft—an approach that 
could require costly rework and retrofits. 


Production Readiness 

The Navy is using the Federal Aviation Administration’s 
air-worthiness certification process, including audits and 
inspections, to demonstrate production readiness. 
Further, the program plans to demonstrate 
manufacturing processes on a pilot production line prior 
to production start. 

Ahead of production start, the contractor continues to 
outfit the four production representative aircraft with VH- 
92A’s unique modifications. According to the contractor, 
despite some parts shortages, it plans to deliver each of 
these aircraft on or ahead of its contractual delivery 
date. 

Program Office Comments 

We provided a draft of this assessment to the 
program office for review and comment. The program 
office provided technical comments, which we 
incorporated where appropriate. The program office 
stated that, with regard to prototype testing, it 
expects to complete testing of a system-level 
integrated prototype in April 2019. 
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Lead Component: Navy 


Common Name: DDG 51 Flight III 



Source: Huntington Ingalls Industries, Pascagoula, MS 


DDG 51 Arleigh Burke Class Destroyer, Flight III (DDG 51 
Flight III) 

The Navy’s DDG 51 Flight III destroyer will be a multi-mission ship 
designed to operate against air, surface, and underwater threats. 
Compared to existing Flight IIA ships of the same class, the new Flight 
III ships will provide the fleet with increased ballistic missile and air 
defense capabilities. Flight Ill’s changes include replacing the current 
SPY-1 D(V) radar with the Air and Missile Defense Radar (AMDR) 
program’s SPY-6 radar and upgrading the destroyer’s Aegis combat 
system. The Navy currently plans to procure 20 Flight III ships. 
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Program Essentials 


Current Status 


Milestone decision authority: Navy 
Program office: Washington, DC 
Prime contractors: Huntington Ingalls 
Industries; General Dynamics-Bath Iron 
Works 

Contract type: FPI (construction) 
Software development approach: 

Mixed 

Next major milestone: Fabrication start 
for second Flight III ship (DDG 126) (April 
2019) 


The Navy and the shipbuilders completed Flight III detail design activities in 
December 2017. As compared to Flight IIA, the Flight III design included 
considerable changes to the ship’s hull, mechanical, and electrical systems 
to incorporate the AMDR program’s SPY-6 radar, and changes to restore 
ship weight and stability safety margins. To reduce technical risk, the Navy 
plans to field all but one—the SPY-6 radar—of the program’s four mature 
critical technologies on other ship classes before integration with Flight III. 

In 2018, however, the Navy identified software-related deficiencies affecting 
SPY-6 that delayed delivery of a radar array for power and integration 
testing with the Aegis combat system by at least 1 year. Despite these 
delays, the Navy plans to complete testing, install the radar on the ship, 
and activate the combat system for shipboard testing by January 2022. 


Estimated Program Cost 

(FY 2019 dollars in millions) 



■ Development ■ Procurement 


Planned Quantities 


20 


■ Development ■ Procurement 



Note: Cost and quantity estimates reflect amounts 
reported in the fiscal year 2019 President’s Budget, 
which are procurement amounts only. 


The Navy expects to complete a draft test and evaluation master plan for 
Flight III by early 2022. The Navy and the Director, Operational Test and 
Evaluation continue to disagree on whether the use of a self-defense test 
ship equipped with Aegis and the SPY-6 radar is necessary to validate 
performance during operational test and evaluation. 

Huntington Ingalls began construction of the first Flight III ship in May 2018 
and plans delivery in fiscal year 2023. Bath Iron Works will begin 
construction of the second Flight III ship in April 2019. Congress provided 
the Navy with multiyear procurement authority, which allows the Navy to 
procure up to 15 Flight III ships on one or more multiyear contracts. In 
September 2018, the Navy awarded multiyear procurement contracts for 10 
Flight III ships: six to Huntington Ingalls and four to Bath Iron Works, with 
options for up to five additional ships split between the yards. The program 
reported that the Navy exercised one of these options in fiscal year 2019 
for an 11th ship. In fiscal year 2019, the Navy added six ships to its planned 
Flight III quantities to work toward its goal of a 355 ship Navy. 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated where appropriate. The program office said that it has 
delivered 67 DDG 51 class ships since its inception in 1985 and the class 
remains in serial production at both new construction shipyards. 
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Lead Component: Navy 


Common Name: FFG(X) 


A 












Guided Missile Frigate (FFG(X)) 

The Navy’s guided missile frigate program is intended to develop and 
deliver a small surface combatant with enhanced lethality and 
survivability as compared to the Littoral Combat Ship (LCS). The Navy 
expects FFG(X) to be an agile, multi-mission ship that provides local 
air defense, maximizes anti-surface and -submarine warfare 
capabilities, and delivers capability to protect and enable 
communications in hostile environments. 


Source: U S Navy 
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Program Essentials 


Current Status 


Milestone decision authority: Navy 
Program office: Washington Navy Yard, 
DC 

Prime contractors (conceptual 
design): Austal USA; Bath Iron Works; 
Ingalls Shipbuilding; Marinette Marine 
Corporation; Lockheed Martin 
Contract types: FFP (conceptual 
design); FPI (detail design and 
construction; planned) 

Software development approach: TBD 

Next major milestone: Preliminary 
design review (June 2019) 


The FFG(X) program continues conceptual design work ahead of planned 
award of a lead ship detail design and construction contract in September 
2020. In May 2017, the Navy revised its plans for a new frigate derived 
from minor modifications of an LCS design. The current plan is to select a 
design and shipbuilder through full and open competition to provide a more 
lethal and survivable small surface combatant. 

As stated in the FFG(X) acquisition strategy, the Navy awarded conceptual 
design contracts in February 2018 for development of five designs based 
on ships already demonstrated at sea. The tailoring plan indicates the 
program will minimize technology development by relying on government- 
furnished equipment from other programs or known-contractor-furnished 
equipment. 


Estimated Program Cost 

(FY 2019 dollars in millions) 



■ Development ■ Procurement 


In November 2018, the program received approval to tailor its acquisition 
documentation to support development start in February 2020. This 
included waivers for several requirements, such as an analysis of 
alternatives and an affordability analysis for the total program life cycle. 
FFG(X) also received approval to tailor reviews to validate system 
specifications and the release of the request for proposals for the detail 
design and construction contract. 

Following the development start decision, the program office will complete 
its evaluation of FFG(X) proposals and plans to award a contract in 
September 2020 to a single shipbuilder for detail design and construction of 
the lead frigate, with options for nine additional ships. 


Planned Quantities 

o 


Attainment of Technology Maturation Knowledge 

As of January 2019 


Conduct competitive prototyping 

O 

Complete technology readiness assessment 

€ 

Validate requirements 

€ 

Complete preliminary design review 

€ 


Not applicable 


■ Development ■ Procurement 

Note: Development costs include fiscal years 2019- 
2023 and procurement costs include fiscal years 
2020-2023, covering 6 of the planned quantities. The 
Navy has yet to identify funding needs beyond 2023. 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office did not have any comments. 


Page 132 


GAO-19-336SP Weapon Systems Annual Assessment 


































Lead Component: Navy 


Common Name: LHA 8 



Source: U S. Navy 


LHA 8 Amphibious Assault Ship (LHA 8) 

The Navy’s LHA 8, third ship of the LHA 6 class of ships, will help 
replace retired LHA 1 Tarawa-class amphibious assault ships. The 
LHA 8 incorporates significant design changes from earlier ships in 
the LHA 6 class to provide enhanced aviation capabilities and a well 
deck that can accommodate two landing craft. The ship is designed to 
transport about 1,350 Marines and their equipment onto hostile 
shores. The LHA 8 is under contract and began construction in 
October 2018. 
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Program Essentials 


Current Status 


Milestone decision authority: Navy 
Program office: Washington, DC 
Prime contractor: Huntington Ingalls 
Industries 

Contract type: FPI (detail design and 
construction) 

Software development approach: 

Mixed 

Next major milestone: LHA 8 ship 
delivery (January 2024) 


Estimated Program Cost 

(FY 2019 dollars in millions) 



■ Development ■ Procurement 


Planned Quantities 


In June 2017, the Navy exercised a contract option for detail design and 
construction of the LHA 8. The LHA 8 incorporates significant design 
changes from earlier ships in the LHA 6 class, but Navy officials were 
unable to quantify the changes. The Navy started construction in October 
2018 and LHA 8 is scheduled to be delivered in January 2024. 

The LHA 8 program office has not identified any critical technologies. 
However, the ship is relying on technology that is currently being developed 
by another Navy program, the Enterprise Air Surveillance Radar (EASR), 
with delivery expected in August 2021. EASR, intended to provide self- 
defense and situational awareness capabilities, is derived from the pre¬ 
existing Air and Missile Defense Radar program, but will be a different size 
and will rotate. LHA 8 program officials have identified the radar as the 
program’s highest development risk. If the radar is not delivered on 
schedule, Navy officials report that this could lead to out-of-sequence 
design and delayed installation and testing. Officials responsible for 
developing the radar, however, stated that the radar is approaching 
maturity and is on schedule to be delivered to the shipbuilder when needed. 

The Navy began construction with about 61 percent of the LHA 8 product 
model completed—an approach inconsistent with shipbuilding best 
practices. These best practices call for 100 percent completion of 3D 
product modeling prior to construction start to minimize the likelihood of 
costly re-work and out of sequence work that can drive schedule delays. 
The Navy, however, estimates that the LHA 8 shipbuilder will not complete 
100 percent of the ship’s 3D product model until June 2019, almost 8 
months after the start of construction. 



■ Development ■ Procurement 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated where appropriate. The program office stated that the Navy 
understands all design changes incorporated on the LHA 8, such as 
reintroducing the well deck and incorporating EASR. According to the 
program office, the Navy does not begin construction on any section of the 
LHA 8 ship before completing that respective section’s design. 
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Lead Component: Navy 


Common Name: LPD 17 Flight II 



Source: Computer Science Corp (CSC) pursuant to 
LX(R) Design Support Contract notional concept LX(R) 


LPD 17 San Antonio Class Amphibious Transport Dock, 
Flight II (LPD 17 Flight II) [formerly LX(R)] 

The Navy’s LPD 17 Flight II program will replace existing amphibious 
ships that the Navy is retiring. The Navy intends to use LPD 17 Flight 
II ships to transport Marines and equipment to support expeditionary 
operations ashore, as well as non-combat operations for storage and 
transfer of people and supplies. The Flight II ships will include a larger 
hull than the retiring ships, but the Navy expects them to provide 
similar capabilities. The Navy plans to develop Flight II incrementally 
over three ships and acquire 13 ships beginning in fiscal year 2019. 
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Program Essentials 


Current Status 


Milestone decision authority: Navy 

Program office: Washington Navy Yard, 
DC 

Prime contractor: Huntington Ingalls 
Incorporated 

Contract type: CPFF (long lead material 
purchasing) 

FPI (detail design and construction) 

CPFF (life cycle and engineering support; 
planned) 

Software development approach: NA 

Next major milestone: Detail design and 
construction contract award (March 2019) 


Estimated Program Cost 

(FY 2019 dollars in millions) 



H Development H Procurement 


Planned Quantities 


The Navy planned to accelerate purchase of LPD 30—the first fully 
configured Flight II ship—after Congress appropriated $1.8 billion above 
the fiscal year 2018 budget request, according to program officials. The 
Navy reported that it awarded contracts in August 2018 for LPD 30 long 
lead time materials and in March 2019 for lead ship construction. 

The Navy based the Flight II design on Flight I, with modifications to reduce 
costs and meet new requirements. According to program officials, roughly 
200 design changes will distinguish the two flights including replacing the 
composite mast with a steel stick. Officials stated that the design would not 
rely on any new technologies. However, the Navy plans to install a new 
radar, the Enterprise Air Surveillance Radar, which is still in development. 
The Navy expects live radar system testing through November 2019, with a 
complete radar prototype in February 2020. Although program officials 
consider these activities to be low risk, the Navy will make its decision to 
begin ship construction by December 2019 without incorporating lessons 
learned from radar testing into the design. Starting construction before 
stabilizing the design could require the Navy to absorb costly design 
changes and rework during ship construction. 

The Navy initially pursued a limited competition for LX(R), but now has a 
non-competitive acquisition strategy for LPD 17 Flight II. The Navy plans to 
award sole-source contracts to Huntington Ingalls—the only shipbuilder of 
Flight I ships—for Flight II construction. Further, the program did not 
request a separate independent cost estimate for Flight II prior to awarding 
the LPD 30 detail design and construction contract. At the same time, the 
Navy identified no plans to establish a cost baseline specific to Flight II. 
Without this baseline, the Navy would report full LPD 17 program costs— 
rather than Flight II specific costs—constraining visibility into Flight II. 


o 

■ Development H Procurement 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated as appropriate The program office stated that LPD Flight II is 
included under the existing LPD 17 acquisition program baseline, and that 
no other viable contractor responded to a public notice regarding the 
Navy’s plan to award Huntington Ingalls the LPD 30 construction contract. 
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Lead Component: Navy 


Common Name: NGJ Low-Band 


Source: U S. Navy. 
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Next Generation Jammer - Low Band (NGJ-LB) 

The Navy’s NGJ-LB is an external jamming pod system that will be 
fitted on EA-18G Growler aircraft. It is expected to replace the ALQ-99 
jamming system and provide enhanced airborne electronic attack 
capabilities to disrupt adversaries’ use of the electromagnetic 
spectrum for radar detection, among other purposes. The Navy plans 
to field a system that jams low-band radio frequencies. The Navy 
plans separate programs for mid- and high-band frequencies. We 
assessed the Low Band program. 
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Program Essentials 


Current Status 


Milestone decision authority: Navy 

Program office: Tactical Aircraft 
Programs, Patuxent River, MD 

Prime contractor: TBD 
Contract type: TBD 

Software development approach: TBD 
Next major milestone: Completion of 
two demonstration of existing 
technologies contracts (June 2020) 


Estimated Program Cost 

(FY 2019 dollars in millions) 



The Navy is determining the acquisition strategy for the NGJ-LB program 
and analyzing potential solutions to meet its capability needs. In October 
2018, the Assistant Secretary of the Navy (Research, Development, and 
Acquisition) directed the NGJ-LB program manager to proceed with 
planning to execute the program as a middle tier acquisition. A middle tier 
acquisition—also referred to as a Section 804 program—is a program that 
uses a streamlined acquisition process to rapidly prototype or field 
capabilities within a 5-year period. 

In October 2018, the program awarded two demonstration of existing 
technologies (DET) contracts to assess maturity of technologies, identify 
potential materiel solutions, and inform acquisition strategy development. 
Both contractors—L3 Technologies and Northrop Grumman—are required 
to provide technology demonstration prototypes and demonstrate 
technology maturity in a relevant test environment. Program officials stated 
that the results of these demonstrations will determine whether the program 
will move forward as a middle tier acquisition. According to program 
officials, the Navy plans to conduct a preliminary design review and a 
technology readiness assessment in the future, but will not schedule most 
program events until after the DET contractors complete their work. The 
Navy intends to commence a follow-on development or rapid prototype 
program by the end of fiscal year 2020. 


■ Development ■ Procurement 


Planned Quantities 


To be 

determined 


■ Development ■ Procurement 


Attainment of Technology Maturation Knowledge 

As of January 2019 


Conduct competitive prototyping 

• 

• 

Complete technology readiness assessment 

€ 

Validate requirements 

Complete preliminary design review 

€ 


^^nowiedg^ttained^^^nowiedg^Dlanned^^^^)^^ 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated where appropriate. 
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Lead Component: Navy 


Common Name: P-8A Increment 3 



Source: U S. Navy 


P-8A Poseidon, Increment 3 (P-8A Increment 3) 

The Navy’s P-8A Increment 3 is intended to provide enhanced 
capabilities to the P-8A aircraft in four sets of improvements. The first 
two sets include communication, radar, and weapons upgrades, which 
will be incorporated into the existing P-8A architecture. The following 
sets will establish a new open systems architecture, improve the 
combat system’s ability to process and display classified information, 
and enhance the P-8A’s search, detection, and targeting capabilities. 
DOD made Increment 3 part of the P-8A baseline program in 2016. 
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Program Essentials 

Decision authority level: Navy 
Program office: Patuxent River, MD 
Prime contractor: Various 
Contract type: CPFF (design and 
integration) 

Software development approach: 

Mixed 

Next major milestone: Increment 3 
design review (January 2020) 


Estimated Program Cost 

(FY 2019 dollars in millions) 

$1,195.4 

■ Development ■ Procurement 



Planned Quantities 


Current Status 

The P-8A program has delivered the first set of Increment 3 improvements 
and is scheduled to complete the second set in fiscal year 2019. Since our 
2018 assessment, the P-8A program has delayed the development and 
fielding of the last two sets of capabilities in Increment 3. Specifically, the 
program office reports it deferred events such as platform integration contract 
award, P-8A Increment 3 critical design review, and fielding of remaining 
capabilities by 3 to 11 months. Program officials attributed these new delays 
to receiving 30 percent less in development funds than requested in fiscal 
year 2018. In total, program officials report that limited funding in the last two 
fiscal years has delayed the fielding of the last two sets of capabilities by 12 
to 18 months. 

To upgrade the combat system for P-8A Increment 3, the Navy will integrate 
new hardware and software on the aircraft. The P-8A acquisition program 
baseline provides for 117 aircraft to be retrofitted with Increment 3 
capabilities. According to program officials, P-8A Increment 3 capabilities are 
based on mature technologies. The government will act as the lead integrator 
for the combat system. The combat system upgrade includes an application- 
based open system architecture that will allow the program to compete the 
development and integration of future capabilities. The program office 
conducted a preliminary design review for the combat system upgrade in July 
2018. The program office stated that, among the issues that the review 
identified, it has resolved those associated with certain combat system 
components that exceeded their size, weight, power, and cooling allocations. 
According to program officials, the P-8A Increment 3 critical design review 
will largely focus on integration. 



■ Development ■ Procurement 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated as appropriate. The program office stated that P-8A Increment 3 
has continued to make technical progress to support the combat system's 
critical design review. According to the program office, this progress includes 
closing all technical actions from the preliminary design review conducted in 
July 2018. 
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Lead Component: Navy 


Common Name: SSN 774 Block V 



Source: U S. Navy photo courtesy of Huntington Ingalls Industries. 


SSN 774 Virginia Class Submarine Block V (SSN 774 
Block V) 

SSN 774 is a nuclear-powered attack submarine capable of 
performing multiple missions, with enhanced capabilities for special 
operations and intelligence collection and surveillance. The Navy has 
implemented major upgrades to the class in blocks. The most recent, 
Block V, includes enhanced undersea acoustic improvements called 
acoustic superiority, and increases strike capacity for Tomahawk 
cruise missiles by inserting a new mid-body section called the Virginia 
Payload Module (VPM). 
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Program Essentials 


Current Status 


Milestone decision authority: Navy 
Program office: Washington, DC 

Prime contractor (planned): General 
Dynamics Electric Boat 

Contract type (planned): FPI 

(construction) 

Software development approach: 

Other 

Next major milestone: Block V contract 
award (calendar year 2019) 


Estimated Block V Cost 

(FY 2019 dollars in millions) 



■ Development ■ Procurement 


Planned Quantities 



■ Development ■ Procurement 

Note: In our 2018 report, we reflected Block V 
estimated costs for fiscal years 2019-2022. At that 
time, the Navy had yet to determine funding needs 
beyond fiscal year 2022. The cost estimates above, 
however, reflect information provided by the Navy in 
November 2018, which include funding needs for 
prior years as well as fiscal year 2023. 


In 2019, the Navy plans to award a multibillion dollar, multiyear contract for 
construction of 10 Block V submarines. Under the Navy’s plan, all Block V 
ships will include acoustic superiority improvements, while the VPM will be 
added starting with the second Block V submarine. 

According to program officials, the design of Block V submarines will differ 
from Block IV by approximately 20 percent. Of this 20 percent, the program 
office considers 70 percent to constitute major changes. The program office 
plans to complete basic and functional designs for VPM by construction 
start—an approach consistent with best practices. However, the shipbuilder 
is currently behind schedule in completing detail design work, where (1) the 
design advances to the highest level of fidelity, (2) specific fabrication and 
installation instructions for the shipyard are developed, and (3) required 
production materials are identified. The program now plans to complete 76 
percent of this work by construction start, compared to the 86 percent it 
initially planned, in part due to the shipbuilder’s challenges in using a new 
design tool. Going forward, the Navy and shipbuilder will need to balance 
staffing levels for the remaining Block V design work with design efforts for 
the new Columbia class ballistic missile submarine. Construction of Block V 
and the Columbia class will coincide beginning in fiscal year 2021 .This will 
require the Navy and its shipbuilder to manage staffing demands and other 
resources across both programs. In addition, program officials said vendor 
quality issues with welding on VPM have caused a 3.5-month delay in the 
schedule for the payload tubes for the first two submarines with VPM. The 
Navy plans to recover some time by accelerating tube manufacturing with a 
second vendor, but this approach may increase program costs. 

The Block V effort is subsumed under the SSN 774 major defense 
acquisition program, and is not managed as a separate program. In 2015, 
the Office of the Secretary of Defense shifted the program’s oversight to the 
Navy. SSN 774 had already completed its required defense acquisition 
system milestone reviews before Block V started, but program officials said 
the Navy continues to conduct regular oversight of the Block V. 


Program Office Comments 

In commenting on a draft of this assessment, the program office provided 
technical comments, which we incorporated where appropriate. 
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AIR FORCE 

PROGRAM 

ASSESSMENTS 


GAO Assessed 10 
of the Air Force’s 37 
Current Major Defense 
Acquisition Programs 


The 10 Current Programs GAO 
Assessed Represent 33 Percent 
of the Air Force’s $238 Billion 
Estimated Total Acquisition Costs 

(Fiscal Year 2019 dollars in billions) 


Current program —4 

Current 
programs 

(Not assessed) 

■ We also assessed two future major defense acquisition programs. We did not assess programs that 
had yet to enter technology development or had already progressed into full-rate production. 




Most Current Programs GAO Assessed 
Have Either Cost Growth, Schedule 
Delays, or Both 

Program 

complete 


Total Acquisition Cost Performance 
in Current Programs Varied Among 
Knowledge Points (KP) 

(Fiscal Year 2019 dollars in billions) 


Current Programs Did Not Attain 
Knowledge at Key Points 



• GPS III 


KP1 


114 





-$5.1 

-$0.2 


KP1 to KP2 to Post - 

KP2 KP3 KP3 


KP2 


Oof 9 

current 

programs 

attained 

knowledge 


Oof 8 

current 

programs 

attained 

knowledge 


Programs in system 
development 


B-2 DMS M 


F-15 

EPAWSS #MGUE Inc 1 


o» 


©Space Fence I 


OCX 


KP3 


A cost decrease in one current program—KC-46A- 
accounted for most of the cost decrease between 
KP2 and KP3. 


Oof 4 

current 

programs 

attained 

knowledge 


Average Acquisition Cycle Time of Current Programs Increased by 
25 Months from First Full Estimate (FFE) to Current Estimate 


0 5 10 15 20 25 30 35 40 

Program age 

• Cost growth of more than 15 percent and/or 
schedule delays of more than 6 months 

• Cost growth of 15 percent or less and/or 
schedule delays of 6 months or less 

• Cost growth of 0 percent or less and 
schedule delays of 0 months or less 

Note: Bubble size is based on each program's currently 
estimated future acquisition funding needed. 


Programs' 
average FFE 
cycle time 

Number of months 



Programs' average 
cycle time change from 
FFE to current estimate 


We could not calculate cycle time for GPS III because initial 
capability depends on the availability of complementary systems. 


| Four current programs—APT, GPS IMF, UH-1N | Cost and schedule analysis is primarily based on estimates from December 2017 selected acquisition reports. 

Replacement, and VC-25B—entered system This information may differ from the amounts reported in the individual assessments, which are based on more 

development after December 2017 and therefore recent program estimates in some cases. Please see appendix II for details, 

are excluded from our cost, schedule, knowledge 
attainment, and contract analysis. 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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AIR FORCE 

PROGRAM 

ASSESSMENTS 


Half of the Current and Future Air Force 
Programs Report that Less than 20 
Percent of Planned Acquisition Costs 
are Specifically for Software 


The Air Force was the Milestone Decision Authority in 2017 
for 13 of its Current and Future Programs in GAO’s Review 

Air Force programs 


Percentage of planned acquisition costs 

0 - 20 % 

21-40% I 

81-100% I 

Unknown I 

NA/not I 
tracked I 


USD(A&S) 


programs 




The milestone decision authority 
for APT, FAB-T CPT, GPS III, KC- 
46A, MGUE Inc 1, SDB II, and WSF 
changed from the Under 
Secretary of Defense for 
Acquisition and Sustainment 
(USD(A&S)) to the Air Force in 
2017. 


The Air Force Received One Offer on Solicitations for Just Over Half of its 
Currently Reported Major Contracts for Development and Procurement 


0 2 4 6 8 


Number of programs 


Most Current Air Force Programs 
Report that Less than 40 Percent of 
Work Tasks are Software-Related 

Percentage of work tasks 


0 - 20 % 


21-40% 


81-100% 


Unknown 


NA/not 

tracked 

0 2 4 6 8 


Number of contracts 


2 



Aircraft - Aircraft - C3I Helicopter Munitions Radar Satellite Unspecified 

Bomber Other 


System type 

□ 2 or more offers 



offer 


C3I is short for command, control, 
communications and intelligence 


Number of programs 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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Air Force Program Assessments* 

Page number 

2-page assessments 

Advanced Pilot Training (APT) 

141 

B-2 Defensive Management System Modernization (B-2 DMS-M) 

143 

Combat Rescue Helicopter (CRH) 

145 

F-15 Eagle Passive Active Warning Survivability System (F-15 EPAWSS) 

147 

Family of Advanced Beyond Line-of-Sight Terminals Command Post Terminals (FAB-T CPT) 

149 

Global Positioning System III (GPS III) 

151 

Global Positioning System III Follow-On (GPS IMF) 

153 

KC-46 Tanker Modernization Program (KC-46A) 

155 

Military GPS User Equipment (MGUE) Increment 1 

157 

Next Generation Operational Control System (OCX) 

159 

Small Diameter Bomb Increment II (SDB II) 

161 

Space Fence Ground-Based Radar System Increment 1 (Space Fence Inc 1) 

163 

Utility Helicopter (UH-1N) Replacement 

165 

1-page assessments 0 

B-52 Radar Modernization Program (B-52 RMP) 

167 

VC-25B Presidential Aircraft Recapitalization (VC-25B) 

168 

Weather System Follow-On (WSF) 

169 


a We abbreviate the following contract types in the individual assessments: cost-plus-award-fee (CPAF), cost-plus-fixed-fee (CPFF), cost-plus-incentive- 
fee (CPIF), firm-fixed-price (FFP), and fixed-price incentive (FPI). 

b The VC-25B program has entered system development but released a baseline late in our review, so we assessed it in a one-page format. 
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Lead Component: Air Force 


Common Name: APT 



Source: U S. Air Force. 


Advanced Pilot Training (APT) 

With its APT program, the Air Force is replacing its legacy T-38C 
trainer fleet and related ground equipment by developing and fielding 
newer, more technologically advanced trainer aircraft and an 
associated ground-based training system. The APT program responds 
to the Air Force’s advanced fighter pilot training needs and seeks to 
close training gaps that the T-38C cannot fully address. 



®-®- 


06/23 

Low-rate 
decision 


01-03/24 

End 

operational 

test 




04/25 

Full-rate 

decision 


■© 


10/25 

Initial 

capability 


Program Essentials 

Program office: Wright-Patterson Air 
Force Base, OH 

Prime contractor: Boeing 

Contract type: Indefinite-delivery 
indefinite-quantity with delivery orders: 

FPI (development) 

Software development approach: TBD 

Next major milestone: Design review 
(March 2020) 


Program Performance (fiscal year 2019 dollars in millions) 


First full estimate 
(09/2018) 

Latest 

(09/2018) 

Percentage 

change 

Development 

$1,258.40 

$1,258.40 

0.0% 

Procurement 

$6,782.30 

$6,782.30 

0.0% 

Unit cost 

$23.40 

$23.40 

0.0% 

Acquisition cycle 

85 

85 

0.0% 

time (months) 




Total quantities 

351 

351 

0.0% 


The Air Force approved an acquisition program baseline for APT in September 2018. Total quantities comprise 5 
development quantities and 346 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$1.183.26^^^^ $75.16 

Procurement 
$6.782.26 


Procurement Quantities 

O 

■ Funded to date 

■ To complete 



Attainment of Product Knowledge 

As of January 2019 


Status at Current Status 


Resources and requirements match 

Development 

Start 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

NA 

NA 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

NA 

NA 

• Complete a system-level preliminary design review 

O 

O 

Product design is stable 

Design Review 


• Release at least 90 percent of design drawings 

NA 

NA 

• Test a system-level integrated prototype 

NA 

NA 

Manufacturing processes are mature 

Production Start 


• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

NA 

NA 

• Demonstrate critical processes on a pilot production line 

NA 

NA 

• Test a production-representative prototype in its intended 
environment 

NA 

NA 


^Knowledge attained, ( Knowledge not attained, (information not available, ( Not applicable 
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Common Name: APT 


Lead Component: Air Force 

APT Program 

Technology Maturity 

The APT program office reported that APT does not rely 
on any critical technologies. According to the program’s 
2016 acquisition strategy, the program expected all 
competing vendors to either (1) not employ any critical 
technologies in their proposals or (2) only employ 
critical technologies that were mature and ready to 
demonstrate in a field environment. The strategy also 
said that the program office conducted a preliminary 
technology readiness assessment in 2015 that showed 
the technologies required for the major components of 
the APT—including airframe, propulsion, and ejection 
seat—were mature and demonstrated in operating 
environments similar to that for the APT. The strategy 
also acknowledged, however, that some new APT 
capabilities, such as embedded training systems, 
cockpit displays, and software, might need to be 
developed or integrated during the program’s 
development phase. 

The program did not conduct a preliminary design 
review (PDR) before its September 2018 development 
start. Instead, it plans to hold the PDR in approximately 
June 2019, some 9 months after development start. The 
program received a statutory waiver from the Assistant 
Secretary of the Air Force (Acquisition, Logistics, and 
Technology) that allowed it to delay the PDR. The 
waiver stated that conducting PDR-related activities 
prior to development start would prevent the program 
from meeting national security objectives because it 
would delay transition of pilots to 4th and 5th generation 
fighter aircraft. Further, the waiver noted that holding 
PDR post-development start was low-risk because APT 
would leverage mature systems and technologies and a 
PDR would increase the program’s cost as well as 
delay the program by 12-18 months with “no 
appreciable decrease” in technical risk. However, as we 
have observed in prior work, a PDR conducted prior to 
development start helps ensure a system’s design is 
feasible—which in turn contributes to a match between 
customer needs and available time, funding, and other 
resources. Additionally, we have found that major 
defense acquisition programs that held a PDR prior to 
starting development had fewer schedule delays than 
other programs. 

Design Stability 

The APT program has initiated system design activities, 
but is not currently tracking design drawings. The 
program’s acquisition strategy called for vendors to 
provide flight test data for their systems in order to be 
considered for contract award. The strategy further 
stated that the program office would structure the 
acquistion to require the use of mature systems. 

Looking ahead, the program expects to hold a critical 
design review no later than March 2020. 


Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. 
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Lead Component: Air Force 


Common Name: B-2 DMS-M 



Source: U S. Air Force. 


B-2 Defensive Management System Modernization (B-2 
DMS-M) 

The Air Force’s B-2 DMS-M program plans to upgrade the aircraft’s 
1980s-era defensive management system to a more capable system. 
This system detects and locates enemy radar systems to provide 
threat warnings and avoidance information. This upgrade is expected 
to improve the system’s frequency coverage and sensitivity, update 
pilot displays, and enhance in-flight rerouting capabilities. It will 
improve the reliability and maintainability of the DMS system and the 
B-2’s readiness. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Under 
Secretary of Defense for Acquisition and 
Sustainment 

Program office: Wright-Patterson Air 

Force Base, OH 


First full estimate 
(05/2016) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$1,914.60 

$2,250.40 

+17.5% 

Prime contractor: Northrop Grumman 

Procurement 

$771.6 

$776.50 

+0.6% 

Contract type: FFP (development) 

Unit cost 

$134.31 

$151.34 

+12.7% 

Software development approach: Agile 
development 

Acquisition cycle 
time (months) 

124 

130 

+4.8% 

Next major milestone: Low-rate initial 

Total quantities 

20 

20 

0.00% 

production decision (June 2020) 

The Air Force released a new service cost position for the B-2 DMS-M program in June 2018. Total quantities 
comprise 4 development quantities and 16 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 


$776.46 



Procurement Quantities 


O 

■ Funded to date 

■ To complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Development 

Start 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 



• Test a system-level integrated prototype 

O 

O 

Manufacturing processes are mature 

Production Start 


• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

NA 

NA 

• Demonstrate critical processes on a pilot production line 

NA 

NA 

• Test a production-representative prototype in its intended 
environment 

NA 

NA 


[Knowledge attained, Q Knowledge not attained, ■information not available, Not applicable 
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Lead Component: Air Force 

B-2 DMS-M Program 


Common Name: B-2 DMS-M 


Technology Maturity and Design Stability 

The B-2 DMS-M program has fully matured its critical 
technologies and stabilized its design. The program 
entered system development in March 2016 with four 
critical technologies approaching maturity. Since then, 
Northrop Grumman proposed, and the Air Force 
accepted, an alternative system that provides additional 
capabilities. The alternative system is leveraged from 
other programs and has the same four critical 
technologies. Because the Air Force has tested the 
major subsystems on an existing program, it now 
considers its critical technologies mature. 

The Air Force’s decision to pursue an alternative 
system led to some initial program delays. For example, 
although the Air Force held a preliminary design review 
before entering system development, it completed a 
second preliminary design review for the alternative 
system in March 2018. The program also held its critical 
design review in November 2018, nearly 2 years later 
than originally planned. Consistent with acquisition best 
practices, the program released more than 90 percent 
of its design drawings by the critical design review. 
However, the program did not test a system-level 
integrated prototype—an approach inconsistent with 
acquisition best practices. This decision puts the 
program at risk for costly design changes following 
system-level integration testing if the program identifies 
deficiencies. 

The program continues to face delays and increased 
integration risk in its software development, a critical 
factor for achieving required B-2 DMS-M capabilities. 
The contractor is utilizing an Agile software 
development process for the first time on this program 
and has taken longer than expected to achieve the 
development efficiency to complete the functionality of a 
software block, known as PD 7.1. The program requires 
certification of software block PD 7.1 functionality to 
support developmental flight testing. Program officials 
stated that over the last year the contractor has made 
significant progress in addressing software 
development risk by increasing the number of software 
developers and expanding overtime for existing staff. 
However, software version PD 7.1 may still be at risk of 
not being certified with full functionality by June 2019, 
when the program has scheduled developmental flight 
tests. Lack of software certification could delay these 
tests. Program officials continue to monitor the 
effectiveness of the contractor’s mitigation plans and 
possible schedule delays. 

Other Program Issues 

Program officals report that, in May 2017, due to the 
selection of the alternative DMS system, the program 
office and contractor agreed to an undefinitized contract 
action that changed the development contract from a 


cost-type contract to a firm-fixed-price contract. The 
program office stated that the undefinitized contract 
action included a not-to-exceed amount of $741 million 
until the contract scope and price could be definitized. 
Because there were issues with the contractor 
delivering a complete proposal, the program has 
delayed definitization. The program office reports 
approximately 50 percent of the $741 million not-to- 
exceed amount has been obligated, and officials do not 
expect to definitize the contract action until the spring of 
2019, nearly 2 years after award. 

As program officials began negotiations for using the 
alternative DMS system, they recognized they could not 
execute the related development work within the 
existing Air Force budget. As a result, the Air Force 
conducted a cost-versus-capability trade analysis of the 
program and ultimately decided on a course of action 
that continued with the alternative system but required 
hardware changes on the aircraft. In June 2018, the Air 
Force released an updated service cost position to 
reflect the latest acquisition approach, which identified a 
13 percent unit cost increase over the program’s 
acquisition baseline. This increase was largely due to 
hardware changes and integration updates as a result 
of the increased capabilities of the alternative DMS 
system. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. In reviewing the draft, program 
officials noted that the revised program strategy 
leverages development from other Air Force programs, 
implements additional capability to improve weapon 
system survivability, and provides risk reduction for 
other programs. 

In addition, program officials stated the cost-versus- 
capability analysis ensured that while hardware 
requirements were decreased in order to reduce 
procurement costs, the program will still meet 
performance requirements.They also stated that the 
service cost position now estimates a 12.5 percent 
increase in program acquisition unit cost as well as the 
potential for delays.This estimate includes the increased 
capabilities and associated risk reduction, and is not 
constrained to the not-to-exceed value. 

Program officials also noted that design efforts are 
complete as presented at the critical design review. 
They said that DMS-M installation into the B-2 flight test 
aircraft is underway. According to the officials, the 
critical path to beginning flight test is completion of 
software development and PD 7.1 certification efforts. 
The program office stated that it continues to work 
closely with the prime contractor to achieve schedule 
milestones. 
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Lead Component: Air Force 


Common Name: CRH 



Source: © 2017 Sikorsky, A Lockheed Martin Company. Used with permission for supporting of the 
Air Force’s Combat Rescue Helicopter Program and Combat Rescue Helicopter associated efforts. 


Combat Rescue Helicopter (CRH) 

The Air Force's CRH program will replace the Air Force’s aging HH- 
60G Pave Hawk rescue helicopter fleet. It will provide 113 new air 
vehicles, related training systems, and support for increased personnel 
recovery capability. CRH uses a derivative of the operational UH-60M 
helicopter. Planned modifications to the existing design include a new 
mission computer and software, a higher capacity electrical system, 
larger capacity main fuel tanks, armor for crew protection, a gun mount 
system, and situational awareness enhancements. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Air Force 
Program office: Wright-Patterson Air 
Force Base, OH 

Prime contractor: Sikorsky Aircraft 


First full estimate 
(06/2014) 

Latest 

(12/2018) 

Percentage 

change 

Development 

$2,106.00 

$2,013.80 

-4.4% 

Corporation 

Procurement 

$6,567.50 

$6,240.80 

-5.0% 

Contract type: FPI/FFP (development) 

Unit cost 

$77.67 

$73.66 

-5.2% 

Software development approach: 

Mixed 

Acquisition cycle 

75 

75 

+0.0% 

Next major milestone: Low-rate initial 
production (September 2019) 

time (months) 

Total quantities 

112 

113 

0.9% 


Acquisition cycle time as of November 2018. Cycle time is based on required asset availability date. Initial capability 
is not a formal program milestone. Quantities comprise 10 development quantities and 103 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 


$6,240.83 



Procurement Quantities 



■ Funded to date 

■ To complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ■information not available, Not applicable 
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Lead Component: Air Force 

CRH Program 

Technology Maturity and Design Stability 

The CRH program has matured its one critical 
technology—a radar warning receiver—and stabilized 
its design. At development start in June 2014, however, 
the program had not demonstrated the receiver to final 
form, fit, or function in either a relevant or realistic 
environment—an approach inconsistent with best 
practices. Program officials stated that CRH’s one 
critical technology, the radar warning receiver, was 
proven mature through Integrated Demonstrations and 
Applications Laboratory testing in May 2018 that tested 
based on over 100 simulated real-world threats. As 
indicated by our attainment of production knowledge 
section, we believe that the program cannot 
demonstrate the full maturity of critical technologies until 
they are tested in their realistic environment, such as on 
a helicopter in flight. 

Approximately 72 percent of the CRH design is based 
on the fully operational and fielded UH-60M helicopter, 
while the remainder is based on new design drawings 
that are unique to the CRH. Although the program 
reports that it has released all its planned design 
drawings, the program continues to resolve and test 
fixes for system deficiencies identified in recent testing. 
Program officials noted that, for the identified 
deficiencies—including those that involve the mission 
computer, fuel gauge, and ballistic armor—they are 
either implementing corrections or awaiting testing to 
confirm resolution. These deficiencies may require the 
program to revise currently completed design drawings 
to accommodate necessary changes, which could 
compromise the CRH’s design stability. 

The CRH program also has yet to test a system-level 
integrated prototype to demonstrate its design, although 
best practices criteria state that such a prototype should 
be tested by a program’s critical design review. Instead, 
CRH program officials reported that, in February 2017, 
they began lab-based prototype tests using a partial 
CRH system. However, these tests will not fully 
demonstrate certain CRH subsystems and software, 
which continue to pose technical risk in the program. 

Production Readiness 

In accordance with best practices, program officials plan 
to conduct system-level developmental prototype 
testing and demonstrate critical manufacturing 
processes on a pilot production line prior to the low-rate 
initial production decision, which is planned for 
September 2019. Since our 2018 assessment, the 
program office has delayed planned dates for low-rate 
initial production, the start of developmental testing and 
the helicopter’s first flight. According to program 
officials, problems with suppliers and software 
integration and testing required the program to extend 
the developmental schedule and delay initiating flight 


Common Name: CRH 

tests. The contractor had previously pursued an 
accelerated program schedule. Program officials stated 
that the contractor is now following the program’s 
approved 75-month baseline schedule, due primarily to 
delayed design and supplier performance issues. 
Program officials reported that first flight has changed 
from October 2018 to May 2019 with no changes in 
required or planned testing. 

Program officials stated they do not anticipate 
engineering-related changes to pose a serious risk after 
production begins, but that software development and 
deficiency corrections may be necessary. Program 
officials stated that correcting deficiencies identified 
during testing would not require any increase in the 
government’s program costs. According to these 
officials, the government’s liability is capped at a 
negotiated ceiling price under the fixed-price incentive 
contract for CRH development, and program officials 
project reaching this ceiling. Program officials expect to 
complete integration of CRH’s software and hardware 
prior to the start of operational testing in January 2021. 

Other Program Issues 

Program officials expect to meet CRH affordability 
requirements for average unit cost, but the program’s 
cost estimate for military construction currently exceeds 
its baseline cost constraints by $10 million due to 
increased trainer facilities requirements. In addition, 
program officials reported that this estimate may further 
change due to DOD pricing guide rate changes in the 
future. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. According to the program office, 
CRH has advanced significantly and the contractor has 
mitigated its design and supplier performance issues. 

The program office also said that the contractor 
completed testing of CRH hardware and initial software 
required for-production flight testing in an aircraft- 
representative laboratory. According to the program, 
CRH’s critical technology tested successfully in a 
simulated environment using production hardware. The 
program office stated that all data required to support 
the production decision will be in place. 

The program office also expressed confidence that it 
will achieve its planned dates for production decision, 
subsequent contract award for low-rate initial 
production, and required asset availability. 
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Lead Component: Air Force 


Common Name: F-15EPAWSS 



Source: U.S. Air Force. 


F-15 Eagle Passive Active Warning Survivability System 
(F-15 EPAWSS) 

The Air Force’s F-15 EPAWSS program plans to modernize the F-15’s 
electronic warfare (EW) system used to detect and identify threats, 
employ counter-measures, and jam enemy radars. The program plans 
to reconfigure hardware and software from other military aircraft to 
meet the challenges of today’s EW threat environment. The Air Force 
developed EPAWSS Increment 1 to replace the F-15’s legacy EW 
system. The Air Force has yet to budget for a proposed Increment 2, 
which adds a new towed decoy. We assessed Increment 1. 
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Program Essentials 

Milestone decision authority: Air Force 
Program office: Wright-Patterson Air 
Force Base, OH 
Prime contractor: Boeing 

Contract types: CPIF/CPFF (technology 
maturation and risk reduction) 

CPIF/CPFF/FFP (development) 

Software development approach: 

Mixed 

Next major milestone: Low-rate initial 
production (July 2019) 


Program Performance (fiscal year 2019 dollars in millions) 


First estimate 

■■■■1■■ (11/2016) 

Latest 

(12/2018) 

Percentage 

change 

Development 

$921.10 

$986.20 

7.1% 

Procurement 

$3,546.70 

$3,556.70 

0.3% 

Unit cost 

$10.82 

$11.00 

1.7% 

Acquisition cycle 

83 

82 

-1.2% 

time (months) 




Total quantities 

413 

413 

0.0% 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

“CT 

Procurement 
$3.556.74 



Procurement Quantities 


O 

■ Funded to date 

■ To complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


(Knowledge attained, Q Knowledge not attained, ■information not available, Not applicable 
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Common Name: F-15EPAWSS 


Lead Component: Air Force 

F-15 EPAWSS Program 

Technology Maturity 

Over 2 years after entering development, the EPAWSS 
program reported each of its four critical technologies 
as approaching maturity—a condition inconsistent with 
best practices. The program delayed ground-based 
testing of the critical technologies by several months in 
2018 because of performance shortfalls with some 
components where the technologies reside. The 
remaining technology maturation work also delayed 
software development and other activities needed to 
support flight testing. Specifically, the first flight of an 
EPAWSS-equipped test aircraft is now delayed by 3 
months from December 2018 to March 2019. As a 
result, the Air Force has yet to demonstrate the full 
maturity of EPAWSS technologies. 

Design Stability 

The program reported the release of all EPAWSS 
design drawings, but technology immaturity and the 
redesign of key components pose risk to maintaining 
design stability. For example, the program released only 
87 percent of the EPAWSS drawings at its February 
2017 critical design review (CDR), with the remaining 
drawings released in the months that followed. As a 
result of subsequent redesign efforts to address 
performance shortfalls, the program modified some 
released drawings in the past year. The design changes 
also required a subcontractor to remanufacture 
components of the program’s initial test articles which 
led the program to delay key integration and testing 
activities by several months. 

According to program officials, the components in need 
of redesign performed adequately during prototype 
testing at development start, but did not perform as 
needed during more rigorous testing after CDR. 

Officials indicated that they are still experiencing a 
problem with one of the desired capabilities and may 
need to make additional design changes after the start 
of flight testing. In addition, a subcontractor experienced 
low yield rates for the initial fabrication of a critical test 
article component, leaving a limited number of test 
articles available to support development activities 
before flight testing. As a result, the program plans to 
install EPAWSS hardware without this component on 
several F-15 test aircraft, limiting aircraft functionality 
during the early months of flight testing until the missing 
component is equipped. 

Production Readiness 

The program plans to test a production-representative 
prototype and demonstrate manufacturing processes on 
a pilot production-line before EPAWSS production 
begins.The program schedule, however, indicates 
concurrency between testing and production is 
increasing. For example, the Air Force moved up the 
EPAWSS low-rate initial production decision by one 
month, from August 2019 to July 2019. 


Further, the program is now following a tailored 
approach where its production decision involves two 
separate decision points. The first, in July 2019, 
authorizes the start of EPAWSS low-rate production 
and will be reported as the program’s Milestone C 
decision for statutory purposes. The second decision 
point in 2020 authorizes installation of the already- 
procured hardware on fielded F-15 aircraft. If the 
second decision is delayed, the program may accrue a 
stockpile of unused hardware until installation is 
approved. 

Also as a result of the tailored approach, the program 
will not attain certain pieces of knowledge before it 
starts production. For example, the Air Force decided 
that the EPAWSS program no longer needs to complete 
all hardware qualification testing, conduct an 
operational assessment, or demonstrate full EPAWSS 
performance in-flight prior to the July 2019 production 
decision. The program will instead complete these 
activities in support of the later deployment decision in 
2020. This approach increases the risk that the program 
would need to retrofit already-installed units if system 
testing finds EPAWSS design problems after July 2019. 

Other Program Issues 

Several ongoing modifications to other F-15 aircraft 
subsystems need to be accomplished before EPAWSS 
can be successfully installed on the aircraft. According 
to EPAWSS program officials, these predecessor 
modifications are on schedule to support EPAWSS. In 
addition, the Air Force now plans to procure EPAWSS 
for both the F-15C and F-15E after reporting last year 
that only the F-15E would be upgraded. As a result, the 
program increased its current cost estimate over the 
past year, but it is now in keeping with the first full 
estimate, which included both F-15 models. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that to 
expedite replacement of the functionally obsolete EW 
self-protection system on F-15s today, the EPAWSS 
acquisition heavily leverages mature hardware from 
existing U.S. and foreign military EW programs. The 
program office indicated that availability of critical 
hardware components unique to F-15 EPAWSS drives 
the program schedule. The program office maintained 
that the system’s design is now stable and noted that it 
expects to begin flight tests by early summer 2019. The 
program office contended that the technical maturation 
of EPAWSS components will help support a decision to 
purchase a small quantity of production hardware 
before development is complete to help accelerate 
initial operational capability. The program office stated 
that while this strategy is not without risk, it provides the 
Air Force an opportunity to transfer operational risk from 
F-15 aircrews to the acquisition community. 
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Lead Component: Air Force 


Common Name: FAB-T CPT 



Source: U S. Air Force. 


Family of Advanced Beyond Line-of-Sight Terminals 
Command Post Terminals (FAB-T CPT) 

The Air Force's FAB-T program plans to provide a family of satellite 
communication terminals for airborne and ground-based users to 
replace legacy communication terminals. In July 2015, DOD separated 
FAB-T into two subprograms: command post terminals (CPT), which 
we assess here, and force element terminals (FET). CPT is expected 
to provide communications for nuclear and conventional forces through 
ground command posts and E-6 and E-4 aircraft. FET is expected to 
provide capabilities for B-52 and RC-135, at a minimum. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Air Force 
Program office: Bedford, MA 


First full estimate 
(12/2007) 

Latest 

(10/2018) 

Percentage 

change 

Prime contractor: Raytheon 

Contract type: FPI (development) 

Development 

$739.20 

$1,239.30 

+67.6% 

FFP (production) 

Procurement 

$993.50 

$619.10 

-37.7% 

Software development approach: 

Unit cost 

$18.24 

$17.05 

-6.5% 

Incremental 

Acquisition cycle 

129 

225 

+74.4% 

Next major milestone: Start of 
operational testing (August 2019) 

time (months) 

Total quantities 

95 

109 

+14.7% 


Latest estimate reflects amounts from the Defense Acquisition Executive Summary following the new acquisition 
program baseline in July 2018. Total quantities comprise 25 development quantities and 84 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 

$1,239.29 


o 


Procurement 


$107.73 


$511.22 

O 


Procurement Quantities 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


Funded to date 
| To complete 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


(Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Common Name: FAB-T CPT 


Lead Component: Air Force 

FAB-T CPT Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The FAB-T CPT program has several critical 
technologies, half of which continue to approach 
maturity, and reports a stable design. However, until the 
program fully matures all of its critical technologies— 
and demonstrates each of them in a final form, fit, and 
function within a realistic environment—current design 
stability remains at risk of disruption. The program has 
also entered low-rate initial production, but has yet to 
demonstrate statistical control of critical manufacturing 
processes—an approach inconsistent with best 
practices. 

FAB-T CPT is comprised of five terminal configurations, 
some of which have different critical technologies. Two 
modification kit terminals adapt existing antennas for 
ground fixed and airborne platforms and are fully 
functional but will eventually be retrofitted with new 
antennas. The other three configurations use new 
antennas for ground fixed, ground transportable, and 
airborne platforms. According to the program office, the 
four configurations currently in production—the two 
modification kit terminals and the new antenna ground 
fixed and transportable terminals—are now mature. The 
program previously planned for all the new antenna 
terminals to be fully mature by July 2016, but delays 
continue in developing the final configuration, the new 
airborne antenna. 

The program office now expects to achieve initial 
operational capability by June 2021—an 18-month 
delay from the December 2019 date it previously 
reported—and full operational capability by March 
2023—a 15 month delay from the December 2021 date 
it anticipated. The program’s revised acquisition 
program baseline was approved in July 2018. 

The program reported that, as of October 2018, it 
intends to procure approximately 63 percent of FAB-T 
total units during low-rate production. The program’s 
purchase of such a quantity corresponds with 2015 
direction from the Under Secretary of Defense for 
Acquisition, Technology, and Logistics, which 
authorized the FAB-T program to purchase more than 
60 percent of its total units during low-rate production. 
Programs generally must provide a rationale to 
Congress if low-rate production quantities will exceed 
10 percent of total planned procurement quantities. 
FAB-T officials said that the higher number of units are 
required to demonstrate initial operational capability due 
to the various configurations and platforms needed. 
Officials said they plan to request permission to procure 
additional terminals before a full-rate procurement 
decision to take advantage of cost control opportunities 
from volume discounts. With the majority of terminals 
already approved, officials noted that initial operational 
testing would not significantly inform a full-rate decision. 
We have previously found that beginning production 


before demonstrating that a system will work as 
intended increases the risk of needing substantial 
design changes and costly modifications to already- 
produced systems. 

The FAB-T program reported that its manufacturing 
processes are nearing maturity. Officials also reported 
the program has contracted for a total of 51 terminals to 
date: 27 modification kit terminals, 15 new antenna 
ground fixed terminals, and 9 ground transportable 
terminals. As of December 2018, the contractor has 
delivered 22 terminals, and the program has installed 5 
to begin testing. 

Other Program Issues 

The program office is developing the acquisition 
strategy for FET, its other subprogram, as a middle tier 
acquisition—also referred to as a Section 804 
program—which uses a streamlined acquisition process 
to rapidly prototype or field capabilities within a 5-year 
period. The program expects the Air Force to approve 
this strategy in February 2019 and to then award a 
development contract by June 2019. Until the FET 
subprogram is executed, FAB-T cannot achieve its 
planned capabilities that are based on the interaction of 
bomber aircraft with intelligence, surveillance, and 
reconnaissance aircraft and CPTs. 

FAB-T is designed to communicate through the 
Advanced Extremely High Frequency (AEHF) network 
of satellites, four of which have already been launched. 
As FAB-T delays mount, the first launched AEHF 
satellite might near the end of its projected 14-year 
operational lifespan by the time FAB-T is available. All 
six AEHF satellites are expected to be on-orbit before 
FAB-T is operational. The lack of synchronization 
between the two programs has resulted in the 
underutilization of costly satellite capabilities. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated as 
appropriate. The program office stated that it faced 
challenges resolving issues identified during testing and 
finalizing operational software, which delayed the 
completion of operational testing by 5 months. 

According to program officials, the program is still on 
track to complete operational testing by November 
2019. Program officials said they installed the first 
operational terminal in January 2019 and plan to 
complete development of the new airborne antenna 
configuration by September 2019. However, the 
program office also noted that it is pursuing options to 
address significant funding shortfalls that could affect 
test and terminal procurement activities. According to 
program officials, the Air Force designated the FET 
subprogram as a middle tier acquisition, but the 
acquisition strategy has not yet been approved. 
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Lead Component: Air Force 


Common Name: GPS III 



Global Positioning System III (GPS III) 

The Air Force's GPS III program is building and fielding a new 
generation of satellites to supplement and eventually replace GPS 
satellites currently in use. GPS III will provide a stronger military 
navigation signal, referred to as M-code, to improve jamming 
resistance, and will provide a new civilian signal that will be 
interoperable with foreign satellite navigation systems. Other programs 
are developing the related ground system and user equipment. 
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Program Essentials 

Milestone decision authority: Air Force 
Program office: El Segundo, CA 
Prime contractor: Lockheed Martin 
Contract types: CPAF (development) 
CPAF/FPI (production) 

Software development approach: 

Mixed 

Next major milestone: Start of 
operational testing (January 2020) 


Program Performance (fiscal year 2019 dollars in millions) 


First full estimate 
(05/2008) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$2,826.50 

$3,436.20 

+21.6% 

Procurement 

$1,587.00 

$2,381.40 

+50.1% 

Unit cost 

$551.68 

$581.76 

+5.5% 

Acquisition cycle 

N/A 

N/A 

N/A 

time (months) 




Total quantities 

8 

10 

+25.0% 


We could not calculate GPS III cycle times because the initial capability depends on the availability of 
complementary systems. Total quantities comprise 2 development quantities and 8 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

“XT 

Procurement 

$305.15 



Procurement Quantities 



■ Funded to date 
MTo complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Common Name: GPS III 


Lead Component: Air Force 

GPS III Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The GPS III program office reported that all eight of its 
critical technologies are mature and that the design is 
stable. Lockheed Martin has delivered two of the 10 
GPS III satellites to the Air Force, while the remaining 
eight are in various stages of production. In September 
2017, the Air Force declared that the first GPS III 
satellite was available for launch. This declaration 
followed an Air Force investigation into performance 
anomalies observed in a propulsion subsystem 
common to multiple Air Force programs. Although the 
Air Force did not determine the root cause of the 
anomalies, it tested multiple GPS III propulsion 
subsystems and was satisfied with the performance. As 
a result, the program office certified that the first GPS III 
satellite met all documented requirements. While 
Defense Contract Management Agency officials 
indicated that there were no design changes to the 
subsystem, program officials stated that the Air Force 
introduced changes to how ground control operators fire 
the subsystem to raise a satellite into orbit and that the 
subsystem manufacturer introduced measures to 
ensure greater uniformity in the manufacturing process. 

The Air Force declared in August 2018 that the second 
satellite was available for launch, and the third satellite 
will be available for launch in May 2019. After prior 
problems with delayed deliveries of navigation payload 
components, the program office has noted 
improvements in the navigation payload subcontractor’s 
management and schedule performance over the past 
year. The program office continues to face delayed 
deliveries of certain other satellite components, such as 
a power regulation unit and the propulsion subsystem. 
The program office is working to reduce potential effects 
of propulsion subystem delivery delays by temporarily 
installing a non-flight propulsion subsystem on the 
affected satellites. This step allows the program to 
progress with assembly, integration, and testing while 
awaiting flight unit deliveries. 

Other Program Issues 

The Air Force launched the first GPS III satellite on 
December 23, 2018, on a SpaceX Falcon 9 rocket. In 
preparation for the launch, the Air Force conducted a 
series of mission readiness events, culminating in a 
final mission rehearsal in November. Next, the Air Force 
will conduct 6 months of on-orbit testing, including 
testing the satellite’s non-navigation functions and 
turning on its navigation payload, according to program 
officials. The Air Force will not conduct operational 
testing of the navigation payload, however, until it 
modifies the Operational Control Segment (OCS) to 
operationally control the payload. That modification is 
currently scheduled to be fielded in October 2019. 
Afterward, the Air Force will begin 6 months of 


additional testing, including operational testing of the 
modified OCS with the GPS III satellite for all currently 
available GPS navigation signals beginning in January 
2020. The Air Force plans for a second OCS 
modification in early 2020 to enable some of the GPS III 
M-code capabilities. 

Because of delays to the GPS Next Generation 
Operational Control System (OCX)—needed to enable 
the full range of GPS III capabilities—the GPS III 
program expects to accept delivery of at least the first 
nine satellites before beginning developmental and 
operational testing with OCX Block 1. The Air Force 
anticipates that these tests, planned to begin in 2023, 
will confirm GPS Ill’s modernized signal capabilities. 
This sequencing, however, introduces the possibility 
that testers will discover deficiencies to already 
produced or launched satellites—thus constraining the 
Air Force’s corrective options—and carries risk to 
overall GPS III cost, schedule, and performance. 

Program Office Comments and GAO Response 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office indicated that it 
is focused on GPS III production, launch, and mission 
operations. It stated that the first GPS III satellite 
successfully launched on December 23, 2018, and 
continues to progress through on-orbit checkout and 
testing. The program office also said the second GPS III 
satellite recently arrived at the Astrotech Space 
Operations facility in Titusville, Florida, in preparation 
for a planned July 2019 launch. According to the 
program office, the third GPS III satellite is on track to 
be available for launch in May 2019, and satellites 4 
through 10 are progressing through various stages of 
production. In addition, the program office stated that 
GPS Ill’s production is currently “robust” and has 
overcome past technical problems. We plan to seek 
further detail on the program’s manufacturing metrics as 
part of our next annual assessment. 
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Lead Component: Air Force 


Common Name: GPS IMF 



Source: Lockheed Martin Corporation. 


Global Positioning System III Follow-On (GPS IMF) 

The Air Force's GPS MIF program will build upon the efforts of the GPS 
III program to develop and field next generation GPS satellites to 
modernize and replenish the GPS satellite constellation. In addition to 
the capabilities built into the original GPS III design, GPS IMF will 
provide new capabilities. These include a steerable, high-power 
military code (M-code) signal, known as Regional Military Protection, 
to provide warfighters with greater jamming resistance in contested 
environments. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Air Force 
Program office: El Segundo, CA 


First full estimate 
(09/2018) 

Latest 

(09/2018) 

Percentage 

change 

Prime contractor: Lockheed Martin 

Contract types: FPI (development) 

Development 

$3,214.50 

$3,214.50 

0.00% 

Software development approach: 

Procurement 

$6,216.80 

$6,216.80 

0.00% 

Mixed 

Unit cost 

$428.70 

$428.70 

0.00% 

Next major milestone: Design review 
(March 2020) 

Acquisition cycle 
time (months) 

N/A 

N/A 

N/A 


Total quantities 

22 

22 

0.00% 


We could not calculate cycle time because initial capability depends on the availability of complementary systems. 
Total quantities comprise 2 development quantities and 20 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 

$2.986.26 ^ __ $22 8.2 2 


o 


Procurement 


$6.216.83 



Procurement Quantities 



■ Funded to date 
MTo complete 


Attainment of Product Knowledge 

As of January 2019 


Status at Current Status 


Resources and requirements match 

Development 

Start 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• 

• 



• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

NA 

NA 

• Complete a system-level preliminary design review 

NA 

NA 

Product design is stable 

Design Review 


• Release at least 90 percent of design drawings 

NA 

NA 

• Test a system-level integrated prototype 

NA 

NA 

Manufacturing processes are mature 

Production Start 


• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

NA 

NA 

• Demonstrate critical processes on a pilot production line 

NA 

NA 

• Test a production-representative prototype in its intended 
environment 

NA 

NA 


Knowledge attained, ( Knowledge not attained, (information not available, ( Not applicable 
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Lead Component: Air Force 

GPS IMF Program 

Technology Maturity 

The GPS IMF program office reports that its two critical 
technologies—an L-band traveling wave tube amplifier 
(TWTA) and a digital waveform generator—are mature 
to the level generally required for the program to begin 
development. The Air Force certified that the program 
has demonstrated the system’s technologies in a 
relevant environment prior to development start. 

The TWTA is designed to provide the signal 
amplification required for the GPS IMF satellite’s 
regional military protection capability. The Air Force’s 
Space and Missile Systems Center (SMC) conducted 
an independent assessment prior to the program’s 
development start and found that the TWTA provides 
lower technical risk and is more mature than the 
alternative amplifier the program office had previously 
considered. Program officials note thatTWTAs have 
proven performance on orbit in other DOD space 
programs, such as the Advanced Extremely High 
Frequency satellite. Although the TWTAs have not been 
proven in space at the power level required for GPS 
IMF, SMC reports that the contractor has successfully 
tested an engineering model in a representative 
environment. 

The GPS IMF navigation payload will incorporate a new 
digital waveform generator to produce the military and 
civil waveforms, or signals, that provide the basis of 
GPS navigation. SMC assessed the digital waveform 
generation as a low-risk function, but noted that 
integrating the navigation signal functions into a single 
electronics box presents a higher risk. Of note, 
integrating signal functions proved difficult in developing 
the GPS III program’s navigation payload. The 
contractor demonstrated the digital waveform generator 
with hardware that was approaching maturity and which 
SMC assessed as being close to flight design. Program 
officials noted that the shift from the partially analog 
navigation payload of GPS III to the completely digital 
payload of GPS IMF streamlines aspects of the 
manufacturing process. 

Design Stability 

The Air Force waived the requirement that the program 
conduct a preliminary design review prior to 
development start, in part to expedite a contract award 
given DOD’s critical national security need for the GPS 
IMF’s unique capabilities. As part of required 
certifications at development start, the Air Force 
certified that GPS IMF had already demonstrated a level 
of technological maturity beyond preliminary design 
review. According to the Air Force, the program 
demonstrated this level of maturity in the production 
readiness feasibility work it conducted between May 
2016 and June 2017. However, we have found that 
completing a preliminary design review prior to 


Common Name: GPS IMF 

development start is an acquisition best practice 
associated with lower cost and schedule growth. 

The GPS IMF program schedule plans for the contractor 
to deliver a production-ready satellite design soon after 
contract award. The program office is currently working 
toward a critical design review in spring 2020 to 
demonstrate design stability in advance of a production 
decision, planned for 6 months later. According to 
program officials, the GPS IMF satellite will heavily 
leverage mature technologies from GPS III, primarily in 
the satellite bus design. The program plans to test a 
system-level integrated prototype that includes all key 
subsystems and components, but with less redundancy 
than the final configuration, prior to integrating and 
testing the first GPS IMF satellite. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. According to the 
program office, since the Air Force awarded Lockheed 
Martin the fixed-price-type contract for GPS III Follow- 
On satellites in 2018, the program has been working 
closely with the contractor to validate contractor delivery 
milestones to ensure no schedule growth occurs. The 
program office also reported that it recently completed 
an integrated baseline review in March 2019 and is 
currently pursuing a critical design review to validate a 
production-ready satellite design. 
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Lead Component: Air Force 


Common Name: KC-46A 



Source: © 2016 Boeing Company - Photo by Paul Weatherman 


KC-46 Tanker Modernization Program (KC-46A) 

The Air Force’s KC-46A program plans to convert an aircraft designed 
for commercial use into an aerial refueling tanker for operations with 
Air Force, Navy, Marine Corps, and allied aircraft. The program is in 
the first of three planned phases to replace roughly a third of the Air 
Force’s aging aerial refueling tanker fleet, comprised mostly of KC- 
135s. The Air Force has designed the KC-46A to improve on the KC- 
135’s refueling capacity, efficiency, and capabilities for cargo and 
aeromedical evacuation, and to integrate defensive systems. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Air Force 
Program office: Wright-Patterson Air 
Force Base, OH 

Prime contractor: Boeing 


First full estimate 
(02/2011) 

Latest 

(01/2019) 

Percentage 

change 

Development 

$7,671.90 

$6,313.80 

-17.7% 

Contract types: FPI (development) 

Procurement 

$37,253.70 

$31,450.10 

-15.6% 

FFP (production) 

Unit cost 

$274.12 

$225.76 

-17.6% 

Software development approach: 

Mixed 

Acquisition cycle 

78 

112 

+43.6% 

Next major milestone: Start of 
operational testing (May 2019) 

time (months) 

Total quantities 

179 

179 

0.0% 


Total quantities comprise 4 development quantities and 175 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

“XT 

Procurement 

S21.910.58j. $9.539.50 

u 


Procurement Quantities 



■ Funded to date 

■ To complete 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


(Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Lead Component: Air Force 

KC-46A Program 


Common Name: KC-46A 


Technology Maturity and Design Stability 

The KC-46A’s three critical technologies—two software 
modules related to situational awareness and a display 
that allows the crew to monitor aerial refueling—are fully 
mature. 

At its 2013 critical design review (CDR), the program 
had released over 90 percent of design drawings. 
However, as reflected in our attainment of product 
knowledge graphic, we lack insight into design stability 
from CDR forward because (1) the discovery of aircraft 
wiring deficiencies post-CDR and consequent re-design 
of the wiring system called into question the accuracy of 
the design drawing information available at CDR, and 
(2) after CDR, the program stopped using drawings to 
assess design and instead began using other methods 
to measure design status. 

Based on developmental testing, the Air Force identified 
three critical refueling deficiencies that require design 
changes. First, refueling operators are not aware of 
instances when the boom strikes receiver aircraft. 

These strikes are due to a shortcoming with the system 
that provides information to their display screens and 
may damage the low observable aircraft coating on 
some receivers, possibly making stealth receiver aircraft 
visible to radar. Second, according to program officials, 
the aerial refueling operator’s screen does not provide 
sufficient visual sharpness and adaptation to changing 
background and lighting to allow for safe refueling in all 
environmental conditions. The officials stated Boeing 
will make software or hardware changes related to 
these two deficiencies without cost to the government. 
Third, program officials reported that the boom remains 
too stiff during refueling attempts with lighter receiver 
aircraft, which could cause it to strike the receiver 
aircraft. Boeing is working to resolve this issue, but 
program officials said the Air Force will have to fund 
implementation of any corrective fixes because the 
system was designed to an international standard and 
the Air Force needed it to be designed to a lower 
stiffness standard. Program officials estimate that it will 
take 3 to 4 years to correct these deficiencies. 

As of February 2019, Boeing has completed 92 percent 
of the development test program. All remaining tests 
relate to the wing aerial refueling pods. Until this testing 
is complete, Boeing may find additional technical issues 
that could require further design changes. 

Production Readiness 

Program officials said that Boeing has not demonstrated 
manufacturing readiness to the desired level due to the 
extension of low-rate initial production. However, they 
said the Federal Aviation Administration has certified 
Boeing’s production process. 

As of February 2019, Boeing has manufactured four 
development aircraft, delivered six low-rate production 


aircraft, and is in the process of producing 40 additional 
low-rate initial production aircraft. Boeing is completing 
the majority of production work on its 767 aircraft 
production line. According to program officials, Boeing 
is completing some production work out of sequence 
due to the earlier wiring and other design problems and 
expects to resolve these by early 2019. The program 
intends to purchase approximately 38 percent of its total 
planned number of production aircraft during low-rate 
initial production. In 2017, the Under Secretary of 
Defense for Acquisition, Technology, and Logistics 
approved this strategy to avoid a break in the 
production line. 

Other Program Issues 

The original development contract required Boeing to 
deliver the first 18 aircraft with nine sets of wing aerial 
refueling pods by August 2017. However, because of 
wiring problems, test and certification delays, and other 
setbacks, the Air Force did not begin accepting aircraft 
until January 2019. As of that time, the Air Force has 
accepted four KC-46A aircraft that are capable of 
refueling other aircraft through two of three types of 
planned refueling options. According to program 
officials, the Air Force is withholding 20 percent of its 
payments to Boeing until Boeing demonstrates that it 
meets contract specifications. Boeing now plans to 
deliver the required aircraft and refueling pods in June 
2020, at which time the aircraft will be capable of all 
three planned types of refueling. Under this plan, 

Boeing will complete work on the development contract 
34 months later than initially planned. 

Program officials negotiated considerations from Boeing 
to account for lost military tanker capability associated 
with the delivery delays, such as obtaining additional 
training at no cost to the government for KC-46A pilots 
and maintenance personnel and support for the aircrew 
training system. Officials reported that the Air Force and 
Boeing finalized the contract modification, including 
these considerations, in December 2018. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. 
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Lead Component: Air Force 


Common Name: MGUE Increment 1 



Si 04/12 09/14 


z Program Preliminary 

O start Design 

w review 


Military Global Positioning System (GPS) User Equipment 
(MGUE) Increment 1 

The Air Force’s MGUE program is developing GPS receivers that will 
be compatible with the military’s next-generation GPS signal, military 
code (M-code). The modernized receiver cards will provide U.S. forces 
with enhanced position, navigation, and timing capabilities, and 
improved resistance to threats, such as jamming efforts by 
adversaries. Increment 1, assessed here, is developing receiver test 
cards for aviation, maritime, and ground platforms. The Air Force 
approved the Increment 2 acquisition strategy in 2018 for development 
of smaller receiver cards and a modernized handheld receiver. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Air Force 
Program office: El Segundo, CA 


First full estimate 
(01/2017) 

Latest 

(06/2018) 

Percentage 

change 

Prime contractors: L-3 Technologies, 
Raytheon, Collins Aerospace 

Development 

$1,555.80 

$1,476.00 

-5.1% 

Contract type: CPIF/CPFF/FFP 

Procurement 

$0.0 

$0.0 

0.0% 

(development) 

Unit cost 

N/A 

N/A 

N/A 

Software development approach: Other 
tailored approach 

Acquisition cycle 

N/A 

N/A 

N/A 

Next major milestone: Start of 
operational testing (February 2020) 

time (months) 

Total quantities 

0 

0 

0.0% 


According to program officials, the latest estimate reflects funding approved to date, but they will seek funding to 
match the approved full estimate. 


Funding and Quantities Attainment of Product Knowledge 

(fiscal year 2019 dollars in millions) As of January 2019 


$345.74 


Development 



Procurement 




■ Funded to date 
MTo complete 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Lead Component: Air Force 

MGUE Increment 1 Program 

Technology Maturity 

The MGUE Increment 1 program office has assessed 
four of its five critical technologies—M-code acquisition 
engine, M-code cryptography, selective availability anti¬ 
spoofing module cryptographic engine, and security 
protection—as fully mature. The program’s fifth critical 
technology, anti-spoof software, is nearing maturity. 
Anti-spoof technology is designed to prevent MGUE 
from acquiring and tracking false GPS signals. Program 
officials said that two contractors have delivered full 
anti-spoof capability, and the third contractor is 
providing incremental capabilities. They noted the 
difficulty of applying maturity standards to MGUE 
because the program is not delivering a final weapon, 
but rather a chip-based receiver card that individual 
programs will integrate into final weapons before it is 
ever fielded in an operational environment. For 
example, to assess maturity for anti-spoof technology, 
the program is using a laboratory simulator as a 
relevant environment because of restrictions regarding 
open air testing of spoofing signals. 

Program protection is included in the contractors’ 
security certification process—a key step for the 
services to procure and integrate receiver cards. 
Program officials said all testing has been completed for 
initial security certification and that two of the three 
contractors have received initial security certification. 
The remaining contractor is expected to complete the 
certification process in spring 2019. 

Design Stability 

We previously recommended that the program 
incorporate a critical design review into its acquisition 
plan to demonstrate the stability of the MGUE design, 
prior to testing the receiver card with the first platform. 
However, DOD formally eliminated this review from the 
acquisition program in January 2017. Program officials 
previously told us the design is stable, and any further 
design problems will be found in ongoing developmental 
testing. In last year’s assessment, we reported that 
contractors delivered final hardware test cards in 2017 
for laboratory tests. Since that time, testing revealed 
deficiencies with the first ground card, which the 
program corrected in 2018. Additionally, the program 
office reports that it modified contracts with each 
contractor to incorporate a performance requirement 
known as “hot start”—the ability of receiver cards to 
transmit legacy signals to existing weapon systems. As 
of December 2018, government testing was ongoing for 
the two contractors’ cards that incorporated this 
capability. The third contractor expects to deliver this 
capability in spring 2019. 


Common Name: MGUE Increment 1 


Production Readiness 

The MGUE program office will not make a production 
decision because the program’s acquisition strategy 
does not include plans to procure cards beyond the final 
test articles. Instead, the program will end with 
operational testing of the first available test cards on 
four, military department-specific weapon systems. 
Integration and testing is ongoing on these systems and 
once completed, expected in April 2021, program 
offices will determine whether to undertake additional 
development and testing to integrate the receiver cards, 
identify their required quantities, and enter into a 
contract for production. While the military departments 
are developing some common solutions, individual 
program offices have the flexibility to pursue their own 
uncoordinated receiver card programs at different times 
and with different contractors. 

Other Program Issues 

The MGUE Increment 1 program is at risk of delaying 
schedule milestones because of delays from 
incorporating the hot start requirement and late 
discovery of hardware and software issues during 
government verification. The program office has taken 
steps to mitigate further delays, such as providing 
additional GPS simulators and test equipment to help 
resolve software problems. The program is also working 
with the Air Force Space Command to determine the 
extent of need for a more comprehensive hot start 
solution that would involve modifying the M-code 
signals, which could drive costs beyond the 
independent estimate. Such a change would also 
require changes to the GPS space and ground 
segments. Additionally, the manufacturer of the chips in 
the receiver cards plans to cease production as early as 
2020, leading some program offices to consider 
procuring the chips in bulk. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. According to the program office, 
MGUE made progress this year demonstrating 
functionality and performance through card-level 
developmental testing of mature cards. The program 
stated that it expects to complete final verification 
testing of the first ground card in March 2019. It also 
said that two cards have met hardware reliability 
requirements, and all cards continue to demonstrate 
capabilities to meet user needs in various 
environments. The program office stated that, within the 
next year, it expects to integrate the final card of each 
type into the military departments’ lead platforms and 
make those cards available to the military departments 
for procurement. 
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Lead Component: Air Force 


Common Name: OCX 



Next Generation Operational Control System (OCX) 

Through its OCX program, the Air Force is primarily developing 
software to replace the existing Global Positioning System (GPS) 
ground control system. The Air Force intends for OCX software to help 
ensure reliable, secure delivery of position, navigation, and timing 
information to military and civilian users. The Air Force is developing 
OCX in blocks that each provide upgrades as they become available. 
We assessed the first three blocks: Block 0, for launch and limited 
testing of new satellites; Block 1, for satellite control and basic military 
signals; and Block 2, for modernized military and additional navigation 
signals. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Under 
Secretary of Defense for Acquisition and 
Sustainment 

Program office: El Segundo, CA 


First full estimate 
(11/2012) 

Latest 

(09/2018) 

Percentage 

change 

Development 

$3,707.60 

$6,231.00 

+68.1% 

Prime contractor: Raytheon 

Procurement 

$0.00 

$0.00 

0.0% 

Contract type: CPIF (development) 
Software development approach: 

Unit cost 

$3,707.60 

$6,231.00 

+68.1% 

Mixed 

Next major milestone: Contractor 

Acquisition cycle 
time (months) 

55 

113 

+ 105.5% 

delivery (February 2021) 

Total quantities 

1 

1 

0.0% 


The Air Force approved a third development acquisition program baseline for the restructured OCX program in 
September 2018. We calculated acquisition cycle time using the program’s initial capability date for Block 2. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$1.879.52 ^ $4.351.52 


o 


Procurement 



Development Quantities 


O 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
| To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Lead Component: Air Force 

OCX Program 


Common Name: OCX 


Technology Maturity and Design Stability 

The OCX program continues software development 
activities necessary to demonstrate system 
performance, but has yet to fully mature the critical 
technologies that underpin the full OCX system. As 
recently as 2017, the program reported that it had 
matured 14 critical technologies. For this and our 2018 
assessment, however, the program identified only five 
critical technologies, which are all nearing maturity, 
because nine of the 14 were delivered with Block 0. The 
OCX program office does not track the metrics we used 
for this assessment to measure design stability, such as 
the number of releasable design drawings, as OCX is 
primarily a software development effort. 

In September 2017, the OCX prime contractor, 
Raytheon, delivered Block 0. In December 2018, Block 
0 successfully supported the Air Force launch of the first 
GPS III satellite. After launch, Block 0 began conducting 
6 months of initial testing on the satellite. As of January 
2019, Raytheon continues development work on Blocks 
1 and 2. Officials report that under the terms of the 
development contract, Raytheon has approximately 2.5 
years remaining to complete development of both 
blocks. 

Since development start, the program has incurred 
persistent cost and schedule growth, which the Air 
Force has attributed to numerous root causes. The 
causes included an unrealistic schedule, 
underestimated costs to fully implement the information 
assurance requirements, and poor contractor and 
government performance. In June 2016, the Secretary 
of the Air Force notified Congress of a critical statutory 
unit cost breach in the program. In October 2016, the 
Air Force restructured the program. Subsequently, the 
Air Force (1) combined delivery of Blocks 1 and 2, set a 
new contractor schedule, and repeated the program 
review associated with system development in June 
2017; and (2) completed a comprehensive baseline 
review in April 2018. In September 2018, DOD 
approved the current baseline, which completed the 
process to put in place new cost and schedule 
estimates. 

Other Program Issues 

Completion of the OCX program within the approved 
baseline requires both (1) timely delivery by the 
contractor and evaluation and acceptance by the Air 
Force, and (2) efficient completion of the planned 7- 
month government-run post-acceptance developmental 
testing before beginning operations that signify the end 
of the program. 

The Air Force reported that the development contract 
with Raytheon ends in June 2021, which our analysis 
has shown is unlikely to provide enough time for the 
contractor to complete OCX development. Independent 


estimates outside of the contractor and program office 
vary on when the contractor will deliver OCX, and when 
the Air Force will subsequently accept OCX, which is 
planned for 2 months later. Independent estimates 
forecast an additional 11 months or more are needed 
past June 2021 to reach acceptance. The program will 
surpass the approved OCX baseline if operations do not 
commence by April 2023. To achieve this milestone, the 
Air Force has to accept OCX delivery 7 months prior in 
September 2022. 

In 2018, the contractor reported it had increased the 
pace of software development activities compared to its 
past performance. Nonetheless, our analysis shows that 
the program faces numerous challenges that could 
negatively affect its cost and schedule. First, significant 
work and technical risk remain in the development plan, 
and the program has taken significantly more time to fix 
software defects than provided for in its schedule. Next, 
implementation of a new software development 
methodology took the program longer than planned and 
subsequently used up two-thirds of the allocated 
schedule reserve. At the same time, Raytheon has 
delayed by more than 1 year its planned decrease of 
OCX staffing levels, which suggests it is struggling to 
achieve the program’s new baseline plan. Finally, our 
analysis shows that the program’s 7-month 
developmental test schedule that occurs after 
acceptance is at risk of doubling in duration because of 
concurrent test events, test plan uncertainty, and risk of 
late discovery of problems. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The OCX program office 
said that, to inform future testing, it is leveraging 
lessons learned from the completion of product testing 
on two areas of software development. Additionally, the 
OCX program office said that it is exploring options with 
the GPS Directorate’s lead development test 
organization to reduce schedule risk to the 7-month 
developmental testing period. These options, it said, 
include conducting a key risk reduction review and 
developmental test at the same time and allowing OCX 
to command some operational satellites before 
accepting OCX from the contractor. 
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Lead Component: Air Force 


Common Name: SDB II 



Small Diameter Bomb Increment II (SDB II) 

The Air Force's Small Diameter Bomb Increment II (SDB II) is a joint 
interest program with the Navy and is designed to provide attack 
capability against mobile targets in adverse weather from extended 
range. It combines radar, infrared, and semi active laser sensors to 
acquire, track, and engage targets. It uses airborne and ground data 
links to update target locations, as well as a global positioning system 
and an inertial navigation system to ensure accuracy. SDB II will be 
integrated with Air Force and Navy aircraft, including the F-15E, F/A- 
18E/F, and F-35. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Air Force 
Program office: Eglin Air Force Base, FL 


First full estimate 
(10/2010) 

Latest 

(10/2018) 

Percentage 

change 

Prime contractor: Raytheon 

Contract types: FPI (development) 

Development 

$1,841.50 

$1,934.30 

+5.0% 

FPI/FFP (low-rate initial production) 

Procurement 

$3,423.00 

$2,571.90 

-24.9% 

Software development approach: Agile 

Unit cost 

$0.31 

$0.26 

-14.4% 

development 

Acquisition cycle 

72 

110 

+ 52.8% 

Next major milestone: Complete 
operational testing (June 2019) 

time (months) 

Total quantities 

17,163 

17,163 

0.00% 


Total quantities comprise 163 development quantities and 17,000 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$386.90 ^ $1.547 38 


n 


Procurement 


$ 2 . 266 . 08^^ ^^ $30587 


15.571 


Procurement Quantities 
1,429 


O 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 


Product design is stable 


Funded to date 
|To complete 


• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


(Knowledge attained, Q Knowledge not attained, ^Information not available, Not applicable 
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Common Name: SDB II 


Lead Component: Air Force 

SDB II Program 

Technology Maturity, Design Stability, and 
Production Readiness 

The SDB II program has matured its critical 
technologies and has stabilized its system design. SDB 
M’s four critical technologies—guidance and control, 
multi-mode seeker, net-ready data link, and payload— 
are mature. However, in 2018 we reported that after 
production start, qualification and flight test failures 
revealed design deficiencies that would require 
hardware and software changes.However, the program 
reports that it has released 100 percent of its design 
drawings, though it does not track revisions to 
previously released drawings. Consequently, we do not 
have visibility into whether design changes have 
disrupted the design stability the program previously 
reported at its January 2011 critical design review. 

To address the deficiencies found in flight testing, the 
program released new software updates to improve 
stationary target performance. As a result, the program 
delayed the start of the operational testing and delayed 
completion of software development to January 2018, 5 
months later than last year’s plans. 

The program office completed Government Confidence 
Tests (GCT) to demonstrate SDB M’s technology and 
design after 5 months of delays. Officials told us that the 
delays were due to software deficiencies and difficulties 
scheduling use of the test range. The program office 
reported that 22 of the 28 test shots were successful. 

Six of the GCTs did not achieve their objective because 
of software deficiencies, most of which involved 
problems with the aircraft relaying inaccurate or 
incomplete information to the weapon. To address the 
six failures, the program office conducted one re-fly test 
and successfully verified the system was ready for 
operational testing. Program officials stated that it 
began operational testing 3 months late because of the 
GCT delays. As of January 2019, officials said the 
program has conducted 49 of its planned 56 operational 
test mission scenarios and estimated completion of the 
remaining tests by June 2019. 

The program expects to declare initial operational 
capability in September 2019, 9 months later than 
anticipated. Despite these testing delays, program 
officials believe that a full-rate production decision in 
September 2022 is still achievable. 

SDB M has been in low-rate initial production since 
March 2017. However, the contractor has yet to bring its 
critical manufacturing process into statistical control. As 
of January 2019, officials said that the program office 
delivered a total of 598 units and reports that the 
contractor is currently producing the remaining 108 
units for Lot 3, which are scheduled for delivery 
completion by August 2019. Officials stated that in 
2018, the program exercised options for production Lots 
4 and 5 for a total of 1,920 units. The Air Force is the 


sole customer for the first three production lots, but 
officials stated that both the Air Force and the Navy will 
procure units under Lot 4 and Lot 5. 

The program currently plans to purchase 58.5 percent 
of total SDB M production quantities during low-rate 
initial production. If a program’s low-rate initial 
production quantity exceeds 10 percent of the total 
production quantity, the program must provide a 
rationale for these quantities in a report to Congress. In 
December 2017, the Air Force reported that SDB II 
exceeded the 10 percent level due to a delay in the 
completion of operational test and evaluation caused by 
schedule revisions to the F-35 program, a threshold 
aircraft. The Air Force further reported that it needed the 
increased quantities to provide production-configured or 
representative articles for operational tests, to establish 
an initial production base for the system, and to permit 
an orderly increase in the production rate for the system 
sufficient to lead to full-rate production following 
operational testing. 

Other Program Issues 

In 2017, the contractor reported cost growth on the first 
three production lots. Program officials said they expect 
this growth to occur in the first five production lots 
because of the contractor’s overly aggressive cost 
proposals early in the system development phase and 
rework of initial production lots. The program stated that 
the government’s liability for cost growth is capped by 
the fixed-price incentive production contract’s terms and 
that the contractor is exploring opportunities to reduce 
production costs. Officials stated that the program has 
not yet negotiated the pricing for Lot 6 and Lot 7, and 
they expect it to be higher than Lots 1-5, which they 
reported were pre-priced options under a competively 
awarded contract. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office 
provided technical comments, which we incorporated 
where appropriate. The program office stated that as of 
April 2019, it has seven operational test mission 
scenarios to complete. According to the program office, 
a January 2019 review of the weapon’s performance in 
operational testing identified deficiencies with the 
weapon datalink communication, and a software update 
is planned for operational test missions in May 2019. 
The program office also said that in February 2019 it 
reported an acquisition program baseline schedule 
deviation for the F-15E required assets available 
threshold date of January 2019, with an estimated 
revised date of September 2019. For Lots 6 and 7, the 
program office said it anticipates a proposal from 
Raytheon by April 30, 2019. 
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Lead Component: Air Force 


Common Name: Space Fence Inc 1 



Space Fence Ground-Based Radar System Increment 1 
(Space Fence Inc 1) 

The Air Force’s Space Fence Inc 1 program is developing a large, 
ground-based radar to detect and track objects in low and medium 
Earth orbit and provide data to a space surveillance network. Space 
Fence will use high radio frequencies to detect and track more and 
smaller objects than previous systems. The Air Force awarded a 
development and production contract for the first radar site in June 
2014. The contract included an option that, if exercised, would enable 
the Air Force to acquire a second site under a separate program. 
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Program Essentials Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Air Force 
Program office: Hanscom Air Force 

Base, MA 

Prime contractor: Lockheed Martin 


First full estimate 
(06/2014) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$1,684.80 

$1,588.6 

-5.7% 

Contract type: FPI (development and 

Procurement 

$0.0 

$0.0 

0.00% 

production) 

Unit cost 

$1,684.77 

$1,588.65 

-5.7% 

Software development approach: 

Incremental 

Acquisition cycle 

124 

124 

0.00% 

Next major milestone: Start operational 
testing (March 2019) 

time (months) 

Total quantities 

1 

1 

0.00% 


Funding and Quantities Attainment of Product Knowledge 

(fiscal year 2019 dollars in millions) As of January 2019 


$19.89 


Development 



Procurement 



^Funded to date 
■To complete 


Status at 


Current Status 


Resources and requirements match 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ■information not available, Not applicable 
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Lead Component: Air Force 

Space Fence Inc 1 

Technology Maturity, Design Stability, and 
Production Readiness 

The Space Fence Inc 1 program has matured its critical 
technologies, stabilized the system design, and 
completed planned production and assembly activities. 
The Space Fence Inc 1 program did not have a 
production start milestone as it is producing only a 
single radar, and thus our production metrics do not 
apply to this program. 

In February 2015, the Air Force completed a technology 
readiness assessment that showed that all seven of the 
program’s critical technologies are fully mature. Space 
Fence’s critical technologies provide capabilities for 
transmitting and receiving radar signals from the radar 
array. 

By its June 2015 critical design review, the program 
attained design stability with release of 100 percent of 
system drawings—a level that exceeds the best 
practices standard. Since early 2016, the contractor has 
tested production-representative hardware and 
operational software on a prototype testbed of the Inc 1 
radar. Program officials stated that while the prototype 
testbed was not a contract requirement, they expect the 
contractor will choose to keep the testbed operational 
throughout the life of the Space Fence program to allow 
fortesting of future upgrades and changes. 

The prime contractor completed production of the radar 
components in February 2018, followed by installation 
and checkout of the radar hardware in March 2018. The 
program identified a production deficiency that affected 
the power supply cabinets. Tests showed that capacitor 
components within the cabinets were failing in testing at 
an unexpectedly high rate, which could have led to 
performance problems had it not been discovered prior 
to the start of operations. To resolve this deficiency, the 
program replaced the capacitors with an upgraded 
design. This problem did not cause any schedule delays 
to the overall installation and checkout of the radar as 
there was sufficient margin in the schedule to cover the 
time needed for the replacement process. 

Other Program Issues 

The Air Force expects Space Fence Inc 1 to provide 
performance sufficient to declare initial operational 
capability. However, the program only plans to declare 
full capability if and when a second radar—identical in 
capability and size to Inc 1—becomes operational. The 
Air Force had included development and production of a 
second radar, which would comprise an Inc 2 program, 
as a contract option. However, officials state that 
because this option was not exercised by August 2018, 
the program likely will need to renegotiate the second 
site pricing, and the renegotiated pricing may be higher 
than what was negotiated at the time of the original 
June 2014 contract award. 


Common Name: Space Fence Inc 1 

Program officials stated that the Air Force has yet to 
budget for an Inc 2 program. According to the program 
office, if the Air Force does not budget for Inc 2 by fiscal 
year 2021, the capability will become significantly more 
costly to acquire. Namely, after fiscal year 2021, the 
program office reports that it will have completed the Inc 
1 program to include reassignment of staff and ramping 
down of the program office. 

Once operational, the Space Fence Inc 1 radar will 
provide a significantly increased number of space object 
observations than previous systems. In part to 
effectively utilize this data, the Air Force attempted to 
acquire new data processing capabilities under its Joint 
Space Operations Center Mission System (JMS) 
program. However, in late 2018, the Air Force ended 
development work on JMS because of numerous 
technical challenges the program encountered. 
Consequently, JMS will not have the capability to 
process Space Fence Inc 1 data. For the near term, the 
Air Force has planned an alternative for completing 
Space Fence operational testing and beginning 
operations. Specifically, the Space Fence program has 
coordinated with the Air Force’s Non-traditional Data 
Pre-Processor (NDPP) program and established a 
method to transfer Space Fence data to existing legacy 
Air Force space situational awareness data processing 
systems. According to Air Force officials, updates to 
these legacy systems will enable them to use some of 
the Space Fence data to perform the space situational 
awareness mission as soon as Space Fence Inc 1 is 
operational. However, to fully utilize what Space Fence 
will provide, the legacy systems will need to be replaced 
by a modernized system. The Air Force hopes to 
complete these modernizations and retire the legacy 
system in December 2020. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. Program officials stated 
that since our assessment, the program has completed 
contractor testing and is preparing to enter 
developmental testing. This represents a 4-month delay 
from previously reported testing dates, with operational 
testing now expected to run from July 2019 through 
September 2019, and initial operational capability (IOC) 
occurring in November 2019. The program noted that 
Space Fence is still on track to meet the IOC threshold 
of January 2020 established in the program’s baseline. 
The program also stated that the system has 
successfully exchanged test messages with NDPP, and 
that some key stakeholders have toured the Space 
Fence site and commended the contractor on the 
readiness of the program. The program said that the 
stakeholders also witnessed system demonstrations 
showing live Space Fence data, which according to the 
program, left the stakeholders satisfied that the system 
is on track to meet mission needs. 
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Lead Component: Air Force 


Common Name: UH-1N Replacement 



Source: U S. Air Force. 


Utility Helicopter (UH-1N) Replacement 

The UH-1N Replacement program aims to replace the Air Force’s 63- 
helicopter fleet, initially manufactured in the 1960s. The UH-1N 
helicopter’s primary missions are securing intercontinental ballistic 
missile sites and convoys and transporting senior government officials 
in the National Capital Region. However, the program office reported 
that the current fleet does not comply with DOD’s nuclear weapons 
security guidance and cannot meet all mission requirements. The 
program plans to acquire 84 helicopters, an integration laboratory, a 
training system, support and test equipment, and associated software. 
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Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority: Air Force 
Program office: Wright-Patterson Air 
Force Base, OH 

Prime contractor: Boeing 


First full estimate 
(09/2018) 

Latest 

(09/2018) 

Percentage 

change 

Development 

$579.10 

$579.10 

0.00% 

Contract type: FFP (integration) 

Procurement 

$2,463.60 

$2,463.60 

0.00% 

Software development approach: TBD 

Unit cost 

$40.06 

$40.06 

0.00% 

Next major milestone: Design review 
(November 2019) 

Acquisition cycle 
time (months) 

60 

60 

0.00% 


Total quantities 

84 

84 

0.00% 


The Air Force approved an acquisition program baseline for UH-1N Replacement in September 2018. Total 
quantities comprise 4 development quantities and 80 procurement quantities. 


Funding and Quantities 

(fiscal year 2019 dollars in millions) 


Development 



Procurement 



Procurement Quantities 


O 

■ Funded to date 
■To complete 


Attainment of Product Knowledge 

As of January 2019 


Status at Current Status 


Resources and requirements match 

Development 

Start 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

NA 

NA 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

NA 

NA 

• Complete a system-level preliminary design review 

NA 

NA 

Product design is stable 

Design Review 


• Release at least 90 percent of design drawings 

NA 

NA 

• Test a system-level integrated prototype 

NA 

NA 

Manufacturing processes are mature 

Production Start 


• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

NA 

NA 

• Demonstrate critical processes on a pilot production line 

NA 

NA 

• Test a production-representative prototype in its intended 
environment 

NA 

NA 


Knowledge attained, ( Knowledge not attained, (information not available, ( Not applicable 
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Lead Component: Air Force 

UH-1N Replacement Program 

Technology Maturity and Design Stability 

The UH-1N Replacement program has identified no 
critical technologies and has begun system integration 
activities ahead of its November 2019 critical design 
review. The UH-1N Replacement program plans to 
integrate an existing helicopter certified by the Federal 
Aviation Administration with previously developed—or 
non-developmental—items. As a result, the program 
office expects to reduce technical risk and facilitate an 
expedited delivery schedule for aircraft. Program 
officials said that through market research, they 
concluded this approach could meet the program’s 
performance requirements and avoid development of 
new technologies. 

In September 2018, the Air Force approved an 
acquisition program baseline and the program entered 
engineering and manufacturing development. Although 
the program is considered non-developmental, the Air 
Force determined that it needed this phase to facilitate 
contractor modifications (integration and testing) to the 
existing helicopter design. The program office 
anticipates that these integration and test activities will 
require up to 3 years and has scheduled a low-rate 
initial production decision for September 2021. 

This schedule, however, is underpinned by the 
assumption that the existing helicopter’s engine and the 
propulsion system meet program requirements. 
Program officials stated that if tests show that either of 
these systems requires additional development or 
modification to achieve certifications, the program office 
will likely have to extend its development phase. Any 
additional development efforts would likely portend 
program delays and cost increases. Officials stated that 
they awarded the UH-1N Replacement contract to 
Boeing (in September 2018) based in part on the 
program’s assessment of Boeings’s ability to provide 
technical components that can be certified through 
testing. 

Production Readiness 

Ahead of the low-rate initial production decision, the 
program plans to establish key metrics for assessing 
the UH-1N Replacement helicopter’s production 
readiness. The program office indicated that it will 
involve the contractor, Boeing, in these future efforts. 
Officials explained that they anticipate Boeing will have 
valuable insight into the appropriate metrics because it 
has already produced the baseline helicopter for 
commercial purposes. Program officials said these 
metrics will be established after critical design review, 
which is planned for November 2019. 

Similarly, the program has yet to determine a timeframe 
for reviewing and establishing software development 
metrics with the contractor, or for completing software 
development and integration. The timing of software 


Common Name: UH-1N Replacement 

development will be important for the program to 
maintain its cost and schedule. We have previously 
found that software development efforts that occur after 
production start place programs at risk of schedule 
delays and cost growth. 

Other Program Issues 

The Air Force Cost Analysis Agency, which produced 
the service cost position for the program, predicted that 
Boeing may lose money on the contract starting in fiscal 
year 2023 and may seek to renegotiate pricing as a 
result. According to the Agency, strict adherence to 
requirements can help the government avoid cost 
increases, particularly when the contractor may seek to 
renegotiate pricing, because changes to contract 
requirements could lead to a contract renegotiation. 

The program office has also not yet determined whether 
the existing helicopter can meet DOD’s cybersecurity 
requirements. Opportunities to change the design to 
implement additional cybersecurity controls are limited 
under the program’s non-developmental item 
acquisition strategy. In addition, program documentation 
indicates that the Air Force may not implement some 
cybersecurity controls. If the helicopter is not able to 
meet cybersecurity requirements, the program office will 
have to determine whether and how to mitigate the 
deficiencies. Program officials said they are 
implementing a cybersecurity risk management 
framework and standing up a cyber security working 
group to understand the specific risks. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The program office did 
not have any comments. 
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Lead Component: Air Force 


Common Name: B-52 RMP 



Source: U S. Air Force. 


B-52 Radar Modernization Program (B-52 RMP) 

The Air Force’s B-52 RMP supports nuclear and conventional 
operations by replacing the current APQ-166 radar on all 76 B-52H 
aircraft in the Air Force inventory. This modernization will allow the Air 
Force to fully utilize the capabilities of the B-52H aircraft to employ an 
array of nuclear and conventional weapons and to perform mission- 
essential navigation and weather avoidance functions. 
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Program Essentials 


Current Status 


Milestone decision authority: Air Force 

Program office: Wright-Patterson Air 
Force Base, OH 

Prime contractor: Boeing 

Contract type: CPFF (risk reduction and 
requirements development) 

Software development approach: 

Mixed 

Next major milestone: Development 
start (September 2020) 


Estimated Program Cost 

(FY 2019 dollars in millions) 



■ Development H Procurement 


Planned Quantities 


The B-52 RMP program has completed the initial subsystem specifications 
for the radar, radome, and crew stations, and is currently defining 
performance requirements in preparation for the development contract 
award in 2020. Additionally, according to program officials, vendors have 
submitted proposals for radar source selection and the program expects to 
make a decision in June 2019. The Air Force plans to modify existing 
training systems, or develop new ones, in support of the B-52 RMP. This 
work will affect all three Air Force Weapon System Trainers (WST)—the 
WST Training Systems Integration Laboratory and both B-52 Offensive 
Station Maintenance Trainers—which are used to help aircrews learn the 
proper techniques and handling of the system. 

The program office plans to release a developmental request for proposal 
in late July 2019. The program has yet to identify any critical technologies, 
but program officials said they are developing a technology readiness 
assessment plan as part of the developmental request for proposal and 
expect to identify candidate technologies as part of that effort by the fourth 
quarter of fiscal year 2019. Consistent with acquisition best practices, the 
program plans to conduct a preliminary design review in July 2020, prior to 
development start in September 2020. 

Attainment of Technology Maturation Knowledge 


As of January 2019 


Conduct competitive prototyping 

o 

Complete technology readiness assessment 

C 

Validate requirements 

• 

Complete preliminary design review 


€ 

^P<nowledg^ttained^^^<nowiedg^ianne^ 

, ] Knowledge not attained, | Information not available, Not applicable 



Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office stated that B-52 RMP is executing its March 
2018 approved acquisition strategy, as planned. 


■ Development H Procurement 
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Lead Component: Air Force 


Common Name: VC-25B 



Source: Boeing Corporation. 


VC-25B Presidential Aircraft Recapitalization (VC-25B) 

Through its VC-25B program, the Air Force is replacing the current 
two VC-25A presidential aircraft with two modified Boeing 747-8 
aircraft. The commercial aircraft will be uniquely modified to provide 
the President of the United States, staff, and guests with safe and 
reliable air transportation with the same level of security and 
communications available in the White House. Aircraft modifications 
will include electrical power upgrades, a mission communication 
system, military avionics, and other systems. 
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Program Essentials 


Current Status 


Milestone decision authority: Under 
Secretary of Defense for Acquisition and 
Sustainment 

Program office: Wright-Patterson Air 
Force Base, OH 
Prime contractor: Boeing 
Contract type: CPFF/CPAF/FFP 
(studies, engineering services, 
commercial aircraft, and development) 

Software development approach: 

Other 

Next major milestone: Design review 
(August 2019) 


Estimated Program Cost 

(FY 2019 dollars in millions) 



The program reported that in January 2016, the Air Force awarded a sole- 
source contract to Boeing for VC-25B risk reduction activities and that it has 
since modified the contract based on the different phases of development 
work. The Air Force purchased two Boeing 747-8 aircraft in 2017. 

Since then, the program has been refining system requirements and 
maturing sub-system designs. The program conducted a system-level 
preliminary design review in October 2018 and established a baseline for 
cost, schedule, and performance in December 2018. We are assessing VC- 
25B in a one-page format this year because the program baseline was 
approved late in our review. We plan to assess it in a two-page format 
going forward. 

According to program officials, the Air Force is incorporating lessons 
learned from the KC-46 program, another commercial derivative aircraft, as 
appropriate. For example, the VC-25B team is implementing a new 
approach to better facilitate wire design, routing and installation processes. 
Program officials stated that they expect to definitize the engineering and 
manufacturing contract by the third quarter of 2019. 

The program office plans to start modification of both aircraft as early as 
2020 after the design is stable. By fiscal year 2024, the Air Force plans to 
accept delivery of aircraft capable to support presidential missions. 


■ Development ■ Procurement 


Planned Quantities 

o 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated where appropriate. 


■ Development ■ Procurement 
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Lead Component: Air Force 


Common Name: WSF 



Source: U S. Air Force. 
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Weather System Follow-On (WSF) 

The Air Force’s polar-orbiting WSF satellite is intended to contribute 
to a family of space-based environmental monitoring (SBEM) systems 
by providing 3 of 11 mission critical capabilities in support of military 
operations. WSF is being developed to conduct remote sensing of 
weather conditions, such as wind speed and direction at the ocean’s 
surface, and provide real-time data to be used in weapon system 
planning and weather forecasting models. The family of SBEM 
systems replaces the Defense Meteorological Satellite Program. 
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Program Essentials 

Milestone decision authority: Air Force 
Program office: El Segundo, CA 

Prime contractor: Ball Aerospace and 
Technologies Corporation 
Contract types: FFP/CPIF (design, risk 
reduction, development, fabrication, 
integration, test, and operations) 

Software development approach: 

Mixed 

Next major milestone: Development 
start (March 2019) 


Estimated Program Cost 

(FY 2019 dollars in millions) 



■ Development ■ Procurement 


Planned Quantities 

2 

(J 

■ Development ■ Procurement 


Current Status 

WSF is to have two payloads: (1) a microwave imager to collect data on 
ocean surface vector wind and tropical cyclone intensity and (2) an 
energetic charged particle sensor to collect space weather data. The Air 
Force awarded a contract in November 2017 for WSF system design, with 
options for development and delivery of up to two WSF satellites, if needed. 
The Air Force plans to start system development in March 2019 and launch 
a satellite in late 2023. 

As a precursor to WSF, the Air Force undertook a technology 
demonstration program using an existing microwave sensor and planned to 
launch it in June 2018. However, the Air Force did not complete the 
planned launch because of problems with the flight software. The Air Force 
is now working toward a potential launch in 2021 to the International Space 
Station. The timely launches of the demonstration and WSF, according to 
the acquisition strategy, are critical to mitigate potential capability gaps. 
Currently, WindSat, a payload operating over 13 years beyond its design 
life, is the only capability that fully meets the service’s needs for ocean 
surface vector wind data. 

The program continues to assess the maturity level of critical technology 
elements. Program officials stated that they plan to complete a technology 
readiness assessment in February 2019, shortly before the program’s 
March 2019 development start. 

Attainment of Technology Maturation Knowledge 


As of January 2019 


Conduct competitive prototyping 

NA 

Complete technology readiness assessment 

€ 

Validate requirements 


Complete preliminary design review 


€ 

• 

Q Knowledge attained, Q Knowledge planned, Q Knowledge not attained, | Information not available, Not applicable 


Program Office Comments 

We provided a draft of this assessment to the program office for review and 
comment. The program office provided technical comments, which we 
incorporated where appropriate. 
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JOINT DOD 
PROGRAM 
ASSESSMENTS 


GAO Assessed 1 of 3 Joint 
DOD-wide Current Major 
Defense Acquisition Programs 


The F-35 Lightning II Program 
Costs are 96 Percent of the $374 
Billion Joint DOD-wide Estimated 
Total Acquisition Costs 

(Fiscal Year 2019 dollars in billions) 


1 Current program 
(F-35 Lightning II) 


Other programs 
(Not assessed) 



Current 

programs 


$14 



| We assessed one current program—F-35 Lightning II. We did not assess programs that had yet to 
enter technology development or had already progressed into full-rate production. 


F-35 Lightning II had Cost 
Growth and Schedule Delays 


Program 

complete 


Total Acquisition Costs for the F-35 
Lightning II Program Increased after 
Knowledge Points (KP) 

(Fiscal Year 2019 dollars in billions) 


F-35 Lightning II Did Not Attain 
Knowledge at Key Points 


KP1 


Oof 1 



$76.7 



• current programs 
attained knowledge 


KP2 

Oof 1 

current programs 
attained knowledge 


KP3 

Oof 1 


• current programs 
attained knowledge 


Programs in system 
development 


F-35 Lightning M’s Acquisition Cycle Time Increased by 62 Months 
from First Full Estimate (FFE) to Current Estimate 


F-35s' FFE 
cycle time 



F-35's cycle time 
change from FFE 
to current estimate 


Number of months 


0 5 10 15 20 25 30 35 40 

Program age 

• Cost growth of more than 15 percent and/or 
schedule delays of more than 6 months 


Note: Bubble size is based on the program’s currently 
estimated future acquisition funding needed. 


■ Cost and schedule analysis is based on estimates 
from the December 2017 selected acquisition 
report. 


■ The Under Secretary of Defense for Acquisition and Sustainment was the milestone 
decision authority in 2017 for F-35 Lightning II. 

■ The F-35 Lightning II program reported that it does not know the percent of planned 
acquisition costs that are specifically for software. 

■ The F-35 Lightning II program reported that it does not know the percent of work 
tasks that are software related. 

■ DOD received single offers for its solicitations for its currently 
reported major procurement contracts for F-35 Lightning II. 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 


Page 171 


GAO-19-336SP Weapon Systems Annual Assessment 



































Joint DOD Program Assessment* 

Page number 

2-page assessment 

F-35 Lightning II (F-35) 

173 


a We abbreviate the following contract types in the individual assessment: cost-plus-fixed-fee (CPFF), cost-plus-incentive-fee (CPIF), and fixed-price 
incentive (FPI). 
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Lead Component: DOD 


Common Name: F-35 



Source: ©2016 Lockheed Martin. 


F-35 Lightning II (F-35) 

DOD is developing and fielding a family of fifth generation strike fighter 
aircraft integrating stealth technologies with advanced sensors and 
computer networking capabilities for the United States Air Force, 

Marine Corps, and Navy, eight international partners, and four foreign 
military sales customers. The family is comprised of three aircraft 
variants. The Air Force’s F-35A variant will complement its F-22A fleet 
and is expected to replace the air-to-ground attack capabilities of the F- 
16 and A-10. The Marine Corps’ F-35B variant will replace its F/A-18 
and AV-8B aircraft. The Navy’s F-35C variant will complement its F/A- 
18E/F aircraft. 
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& 11/96 

z Program 


Q. 
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10/01 

Development 

start 


-© 

02/06 
and 06/07 

Design 

review 


— ® - ® - ® - ® - 

2 06/07 03/12 07/15 and 12/18 

O Production Milestone 08/16 Start 

^ decision recertification USMC and operational 

O USAF initial test 

§[ capability 



01/19 02/19 09/19 

GAO USN Initial End 

review capability operational 
test 


Program Essentials 

Program Performance (fiscal year 2019 dollars in millions) 


Milestone decision authority level: 

Under Secretary of Defense for 

Acquisition and Sustainment 

Program office: Arlington, VA 


First full estimate 
(10/2001) 

Latest 

(06/2018) 

Percentage 

change 

Development 

$43,642.80 

$66,227.30 

+51.7% 

Prime contractor: Lockheed Martin; 

Procurement 

$193,622.50 

$288,950.50 

+49.2% 

Pratt & Whitney 

Contract type: FPI/CPIF/CPFF (aircraft 

Unit cost 

$83.49 

$145.79 

+74.6% 

low-rate initial production) 

Acquisition cycle 

175 

237 

+35.4% 

FPI/CPIF (engine low-rate initial 
production) 

time (months) 

Total quantities 

2,866 

2,470 

-13.8% 

Software development approach: 

Mixed 

Total quantities comprise 14 development quantities and 2,456 procurement quantities. 


Next major milestone: F-35C initial 
capability (February 2019) 






Funding and Quantities 

(fiscal year 2019 dollars in millions) 

Development 

$210,35 $66,016.94 


U 


$209,659.7. 


Procurement 

$79,290.75 


G 


Procurement Quantities 


Attainment of Product Knowledge 

As of January 2019 


Status at 


Current Status 


Resources and requirements match 


Funded to date 
| To complete 


• Demonstrate all critical technologies are very close to final 
form, fit and function within a relevant environment 

• Demonstrate all critical technologies in form, fit and function 
within a realistic environment 

• Complete a system-level preliminary design review 

Product design is stable 

• Release at least 90 percent of design drawings 

• Test a system-level integrated prototype 

Manufacturing processes are mature 

• Demonstrate Manufacturing Readiness Level of at least 9, 
or critical processes are in statistical control 

• Demonstrate critical processes on a pilot production line 

• Test a production-representative prototype in its intended 
environment 


Development 

Start 


[Knowledge attained, Q Knowledge not attained, ^Information not available, J Not applicable 
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Common Name: F-35 


Lead Component: DOD 

F-35 Program 

Technology Maturity and Design Stability 

All of the F-35’s critical technologies are mature and the 
baseline engineering drawings are complete for all three 
aircraft variants. The program office completed the final 
development test flights for the baseline program in 
April 2018, but continues to address over 900 
deficiencies identified with the aircraft’s performance 
prior to the end of development testing. For example, 
the program is developing a new helmet mounted 
display, which will resolve an existing green glow effect 
that can distort a pilot’s vision during night time carrier 
landings. Program officials expect installation of some 
of the new displays in 2019. The program office is also 
testing and integrating software updates to resolve 
other deficiencies, but it did not fully resolve over 800 
other deficiencies prior to the start of operational 
testing. The program obtained a waiver from the Under 
Secretary of Defense for Acquisition and Sustainment to 
start operational testing prior to fully resolving these 
deficiencies. Program officials stated that they expect to 
continue resolving these deficiencies through the start 
of full rate production in October 2019. 

Program officials continue to identify and address 
technical risks, some of which are specific to individual 
variants of the F-35. For example, we reported last year 
that a problem with the F-35’s main fuel throttle valve 
caused the aircraft to move suddenly and without 
stopping until the engine is shut down. In 2018, the 
program implemented software changes to fix this 
problem. Also, across all variants, pilots have reported 
experiencing extreme pressure in the cockpit during 
certain flight maneuvers. Contractor representatives told 
us they have identified the root cause of the excessive 
cockpit pressure and will implement a minor hardware 
change in 2019 to address the issue. Recently, 
following the crash of an F-35B in October 2018, the 
program grounded the F-35 fleet to inspect all of its 
engines. An investigation determined a manufacturing 
defect caused an engine fuel tube to rupture during 
flight, resulting in a loss of power to the engine. The 
program office reported that it identified 117 aircraft with 
the same type of fuel tubes that it must replace. 
According to program officials, the grounding generally 
did not impact the delivery of the aircraft, as the 
contractor has provided replacement fuel tubes that 
were installed on a majority of the affected aircraft by 
the end of 2018. 

Production Readiness 

As of December 2018, the prime contractor has 
delivered 264 production aircraft. Since the start of 
production, F-35 contractors have refined their 
production processes to improve manufacturing 
efficiency and quality. However, the prime contractor 
has identified quality control and late radar deliveries as 
the top production risks in the program. For example, 
because of supplier identified limitations, the prime 


contractor continues to fix gaps between adjacent 
aircraft surface panels attached to the airframe. These 
fixes are needed to meet low observable (stealth) 
performance requirements. The contractor is working 
with its supplier to resolve the problem through 
improved production processes as the program 
approaches its full-rate production decision in October 
2019. 

Other Program Issues 

Following our 2018 assessment, the program delayed 
the start of operational testing by up to 3 months, to 
December 2018. This delay stemmed from software 
upgrades needed to assess the aircraft’s performance. 
To mitigate further delays, the program received 
authorization to complete certain operational tests in 
advance of the formal start of operational testing. For 
example, the program completed cold weather 
operational testing in January 2018. 

Because of evolving threats, the program office 
continues to move forward with Block 4 modernization 
efforts, which will modernize current capabilities and 
develop and integrate new capabilities onto the aircraft. 
In October 2018, the program office updated its 
acquisition strategy, providing a general schedule for 
future technology development and integration. The 
program plans to field new capabilities starting in 
October 2019, but it has yet to complete its acquisition 
program baseline. As a result, the program is 
concurrently testing, producing, and modernizing 
aircraft, which increases the risk of future schedule and 
cost overruns. 

Program Office Comments 

We provided a draft of this assessment to the program 
office for review and comment. The office provided 
technical comments, which we incorporated where 
appropriate. Program officials stated that since the start 
of developmental testing in 2006, more than 2,200 
deficiencies have been discovered and corrected. 
Further, officials reported that, in coordination with the 
warfighting community, they have resolved the highest- 
priority deficiencies and have mitigated the remaining 
deficiencies. Finally, officials commented that aircraft 
deliveries have increased as planned and operational 
testing remains on track for completion in fall 2019. 
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Agency Comments 

We provided a draft of this report to DOD for comment. In its comments, 
reproduced in appendix IX, DOD generally concurred with our 
observations. DOD also provided us with technical comments, which we 
incorporated as appropriate. 


We are sending copies of this report to the appropriate congressional 
committees and offices; the Acting Secretary of Defense; the Secretaries 
of the Army, Navy, and Air Force; and the Director of the Office of 
Management and Budget. In addition, the report will be made available at 
no charge on the GAO website at http://www.gao.gov. 

If you are your staff have any questions concerning this report, please 
contact me at (202) 512-4841. Contact points for our offices of 
Congressional Relations and Public Affairs may be found on the last page 
of this report. Staff members making key contributions to this report are 
listed in appendix X. 

Shelby S. Oakley 

Director, Contracting and National Security Acquisitions 
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List of Committees 


The Honorable James M. Inhofe 
Chairman 

The Honorable Jack Reed 
Ranking Member 
Committee on Armed Services 
United States Senate 

The Honorable Richard Shelby 
Chairman 

The Honorable Richard J. Durbin 
Ranking Member 
Subcommittee on Defense 
Committee on Appropriations 
United States Senate 

The Honorable Adam Smith 
Chairman 

The Honorable Mac Thornberry 
Ranking Member 
Committee on Armed Services 
House of Representatives 

The Honorable Pete Visclosky 
Chairman 

The Honorable Ken Calvert 
Ranking Member 
Subcommittee on Defense 
Committee on Appropriations 
House of Representatives 
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Appendix I: Individual Subcontracting 
Reports in the Electronic Subcontracting 
Reporting System 


Table 18 shows the number of prime contractors for the 82 major defense 
acquisition programs (MDAP) included in our assessment as reported in 
the Electronic Subcontracting Reporting System’s (eSRS) individual 
subcontracting reports for 2018. We reviewed this information against the 
reported prime contract information from the programs’ December 2017 
Selected Acquisition Report (SAR) submissions. 

eSRS individual subcontracting reports include information on contractor 
performance relative to small business subcontracting goals, thus are 
used by the government as a method to monitor small business 
participation. Not all prime contractors for MDAPs are required to submit 
individual subcontracting reports. Instead, some contractors report small 
business participation at a corporate level as opposed to the program 
level and this data is not captured in the individual subcontracting reports. 
In addition, although prime contractors may be required to submit reports, 
they may not yet have done so for the period we reviewed. Their reports 
could also be pending or in rejected status. 


Table 18: Major Defense Acquisition Programs’ Individual Subcontracting Reports in the Electronic Subcontracting Reporting 
System (eSRS) 

Program name 

Number of contracts 
listed in the December 
2017 Selected Acquisition 
Report 

Contracts with an 
accepted individual 
subcontracting report (as 
of July 2018) 

Advanced Arresting Gear (AAG) 

4 

3 

Amphibious Combat Vehicle Phase 1 Increment 1 (ACV 1.1) 

2 

2 

Advanced Extremely High Frequency (AEHF) Satellite 

2 

2 

AGM-88E Advanced Anti-Radiation Guided Missile (AGM-88E AARGM) 

2 

2 

AH-64E Apache New Build (AH-64E New Build) 

2 

2 

AH-64E Apache Remanufacture (AH-64E Remanufacture) 

6 

6 

AIM-9X Block II Sidewinder (AIM-9X Blk II) 

3 

3 

Air and Missile Defense Radar (AMDR) 

1 

1 

Airborne and Maritime/Fixed Station (AMF) 

0 

0 

Armored Multi-Purpose Vehicle (AMPV) 

1 

1 

AIM-120 Advanced Medium Range Air-to-Air Missile (AMRAAM) 

4 

2 

Airborne Warning and Control System Block 40/45 Upgrade (AWACS Blk 
40/45 Upgrade) 

1 

1 

B-2 Defensive Management System Modernization (B-2 DMS-M) 

1 

1 

B61 Mod 12 Life Extension Program Tailkit Assembly (B61 Mod 12 LEP 
TKA) 

2 

1 

C-130J Hercules Transport Aircraft (C-130J) 

4 

4 
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Appendix I: Individual Subcontracting Reports 
in the Electronic Subcontracting Reporting 
System 


Program name 

Number of contracts 
listed in the December 
2017 Selected Acquisition 
Report 

Contracts with an 
accepted individual 
subcontracting report (as 
of July 2018) 

Cooperative Engagement Capability (CEC) 

6 

1 

CH-47F Improved Cargo Helicopter (CH-47F) 

1 

1 

CH-47F Modernized Cargo Helicopter (CH-47F Block II) 

3 

1 

CH-53K Heavy Lift Replacement Helicopter (CH-53K) 

4 

2 

Chemical Demilitarization-Assembled Chemical Weapons Alternatives 
(Chem Demil-ACWA) 

2 

2 

Common Infrared Countermeasure (CIRCM) 

1 

1 

Combat Rescue Helicopter (CRH) 

1 

1 

Gerald R. Ford Class Nuclear Aircraft Carrier (CVN 78) 

4 

4 

DDG 1000 Zumwalt Class Destroyer (DDG 1000) 

1 

1 

DDG 51 Arleigh Burke Class Guided Missile Destroyer (DDG 51) 

4 

4 

E-2D Advanced Hawkeye Aircraft (E-2D AHE) 

4 

4 

EA-18G Growler Aircraft (EA-18G) 

6 

6 

Evolved Expendable Launch Vehicle (EELV) 

9 

2 

F-15 Eagle Passive Active Warning Survivability System (F-15 EPAWSS) 

1 

1 

F-22 Increment 3.2B Modernization (F-22 Inc 3.2B Mod) 

1 

1 

F-35 Lightning II Program (F-35) 

6 

6 

Family of Advanced Beyond Line-of-Sight Terminals (FAB-T) 

2 

1 

Ground/Air Task Oriented Radar (G/ATOR) 

3 

3 

Guided Multiple Launch Rocket System/Guided Multiple Launch Rocket 
Sys Alt Warhead (GMLRS/GMLRS AW) 

4 

4 

Global Positioning System III (GPS III) 

2 

2 

H-1 Upgrades (4BW/4BN) (H-1 Upgrades) 

3 

0 

HC/MC-130 Recapitalization Aircraft (HC/MC-130 Recap) 

1 

1 

Integrated Air and Missile Defense (IAMD) 

1 

1 

Intercontinental Ballistic Missile Fuze Modernization (ICBM Fuze Mod) 

1 

0 

Integrated Defensive Electronic Countermeasures (IDECM) Block 4 

3 

2 

Infrared Search and Track (IRST) 

3 

3 

Joint Air-to-Ground Missile (JAGM) 

1 

1 

Joint Air-to-Surface Standoff Missile (JASSM) 

2 

2 

Joint Direct Attack Munition (JDAM) 

2 

1 

Joint Light Tactical Vehicle (JLTV) 

1 

1 

Joint Precision Approach and Landing System Increment 1A (JPALS Inc 

1 A) 

1 

0 

Joint Tactical Radio System Handheld, Manpack, and Small Form Fit 
Radios (JTRS HMS) 

7 

5 
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Appendix I: Individual Subcontracting Reports 
in the Electronic Subcontracting Reporting 
System 


Program name 

Number of contracts 
listed in the December 
2017 Selected Acquisition 
Report 

Contracts with an 
accepted individual 
subcontracting report (as 
of July 2018) 

KC-130J Transport Aircraft (KC-130J) 

1 

1 

KC-46 Tanker Modernization Program (KC-46A) 

4 

3 

Littoral Combat Ship (LCS) 

2 

2 

Littoral Combat Ship - Mission Modules (LCS Packages) 

1 

1 

LHA 6 America Class Amphibious Assault Ship (LHA 6) 

2 

2 

LPD 17 San Antonio Class Amphibious Transport Dock (LPD 17) 

1 

1 

M88A2 Heavy Equipment Recovery Combat Utility Lift Evacuation System 
(M88A2 HERCULES) 

2 

2 

Military Global Positioning System (GPS) User Equipment Increment 1 
(MGUEIncI) 

3 

2 

Multifunctional Information Distribution System (MIDS) 

4 

4 

MQ-1C Gray Eagle Unmanned Aircraft System (MQ-1C Gray Eagle) 

2 

2 

MQ-4C Triton Unmanned Aircraft System (MQ-4C Triton) 

3 

3 

MQ-8 Fire Scout 

1 

1 

MQ-9 Reaper Unmanned Aircraft System (MQ-9 Reaper) 

4 

3 

Mobile User Objective System (MUOS) 

0 

0 

Next Generation Jammer Increment 1 (NGJ Inc 1) 

2 

1 

Navy Multiband Terminal (NMT) 

1 

1 

Offensive Anti-Surface Warfare Increment 1 (Long Range Anti-Ship 

Missile) (OASuW Inc 1 (LRASM)) 

2 

2 

Next Generation Operational Control System (GPS OCX) 

1 

1 

P-8A Poseidon Multi-Mission Maritime Aircraft (P-8A) 

2 

1 

Patriot Advanced Capability-3 Missile Segment Enhancement (PAC-3 

MSE) 

2 

2 

M109A7 Family of Vehicles (M109A7 FOV) 

2 

2 

Space Based Infrared System High (SBIRS High) 

1 

1 

Small Diameter Bomb Increment II (SDB II) 

3 

3 

Standard Missile-6 (SM-6) 

2 

2 

Space Fence Ground-Based Radar System Increment 1 

1 

1 

SSBN 826 Columbia Class Submarine(SSBN 826) 

2 

2 

Ship to Shore Connector Amphibious Craft (SSC) 

1 

1 

SSN 774 Virginia Class Submarine (SSN 774) 

2 

2 

T-AO 205 John Lewis Class Fleet Replenishment Oiler (T-AO 205 Class) 

1 

1 

Tactical Tomahawk RGM-109E/UGM 109E Missile (TACTOM) 

2 

2 

Trident II (D-5) Sea-Launched Ballistic Missile UGM 133A (Trident II 
Missile) 

9 

7 

UH-60M Black Hawk Helicopter (UH-60M Black Hawk) 

2 

2 
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Appendix I: Individual Subcontracting Reports 
in the Electronic Subcontracting Reporting 
System 


Program name 

Number of contracts 
listed in the December 
2017 Selected Acquisition 
Report 

Contracts with an 
accepted individual 
subcontracting report (as 
of July 2018) 

V-22 Osprey Joint Services Advanced Vertical Lift Aircraft (V-22) 

5 

4 

VH-92A Presidential Helicopter Replacement Program 

1 

1 

Warfighter Information Network-Tactical Increment 2 (WIN-T Inc 2) 

1 

1 

Totals 82 

201 

163 


Source: GAO analysis of Department of Defense (DOD) and eSRS data. | GAO-19-336SP 
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Appendix II: Objectives, Scope, and 
Methodology 


This report includes observations on (1) the cost and schedule 
performance of the Department of Defense’s (DOD) 2018 portfolio of 82 
major weapon programs and the competitive environment in which they 
exist, (2) implementation of key acquisition reform initiatives within 51 
current and future programs, and (3) the knowledge that 51 current and 
future programs attained at key decision points in the acquisition process 


Analysis of Portfolio 
Cost and Schedule 
Performance 


To develop the seven observations on the cost and schedule 
performance of DOD’s 2018 portfolio of current major defense acquisition 
programs (MDAP), we obtained and analyzed cost, quantity, and 
schedule data from Selected Acquisition Reports (SAR) and other 
information in the Defense Acquisition Management Information Retrieval 
system (DAMIR). To assess the reliability of the SAR data, we reviewed 
our previous assessment and DOD officials’ responses regarding any 
changes to DAMIR’s data quality control procedures. We determined that 
the SAR data and the information retrieved from DAMIR were sufficiently 
reliable for the purposes of this report. 


We analyzed this data to determine the number of programs in the 
current portfolio year. We included in the portfolio the programs that 
issued an unclassified December 2017 SAR. The Missile Defense 
Agency’s Ballistic Missile Defense System and its elements are excluded 
from all analyses due to the lack of an integrated long-term baseline. In 
general, we refer to the 82 MDAPs with December 2017 SARs as DOD’s 
2018 or current portfolio and use a similar convention for prior year 
portfolios. 


We entered DAMIR-obtained SAR data into a database and verified that 
the data was entered correctly. We then converted all cost information to 
fiscal year 2019 dollars using conversion factors from DOD Comptroller’s 
National Defense Budget Estimates for Fiscal Year 2019 (table 5-9). 


Our assessment includes comparisons of cost and schedule changes 
over the past year, 5 years, and from baseline estimates, that utilize SAR 
data from December 2017, December 2016, December 2012, and from 
the programs’ initial SAR submissions. 


For our first observation, we compared the 2018 portfolio with the 
programs that issued SARs in December 2016 (2017 portfolio) to identify 
the programs that exited and entered the current portfolio, the total cost 
and number of programs in the current portfolio compared to previous 
years, and the total cost change in the current portfolio over the past year. 
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Methodology 


For the second observation, we used program acquisition start dates to 
calculate individual program ages and the average program age of the 
current portfolio, then compared the results to preceding portfolios dating 
back to 2012. For the third observation, we aggregated DAMIR funding 
stream data for the total planned investment of each portfolio for each 
year since 2008 to determine any trends. We also calculated yearly totals 
for research and development, procurement, and total acquisition cost. To 
distinguish between the funding already invested from the funding needed 
to complete the programs in each portfolio since 2008, we used funding 
stream data obtained from DAMIR for each December SAR submission 
for the years 2007 (2008 portfolio) through 2017 (2018 portfolio). We 
define funding invested as all funding that has been provided to the 
programs in the fiscal year of the annual SAR submission (this includes 
fiscal year 2018 for the December 2017 submission) and earlier, while 
funding remaining is what will be provided in the fiscal years following the 
annual SAR submission (fiscal year 2019 and later for the December 
2017 submission). Invested and remaining research and development 
and procurement funding totals for the 2018 portfolio were organized by 
the military departments overseeing portfolio programs. 

For our fourth observation, we calculated the total acquisition-related cost 
growth of the current portfolio by collecting relevant data from programs’ 
annual SARs. Data from the current (December 2017) and prior year 
(December 2016) SARs was compared and yearly trends analyzed. We 
then divided the programs into percent cost change categories based on 
their total acquisition cost changes over the past year. Finally, we totaled 
the number of programs in each category and the total cost change of the 
programs. Portfolio performance associated with acquisition cost growth 
of its programs was also evaluated against the high-risk criteria discussed 
by DOD, the Office of Management and Budget, and GAO. We calculated 
how many programs attained less than 2 percent increase in total 
acquisition cost over the past year, less than a 10 percent increase over 
the past 5 years, and less than a 15 percent increase from baseline 
estimates. We also calculated the percentage of programs meeting each 
of these high-risk criteria for 2013 through 2018 portfolios to identify any 
changes. For programs with multiple sub-programs presented in the 
SARs, we calculated the net effect of the sub-programs to reach an 
aggregate program result. 

In the fifth observation, we calculated the total acquisition-related cost 
growth and schedule delays of the current portfolio since the programs’ 
first full estimates by retrieving SAR cost data for the 82 programs in the 
portfolio. In addition, we analyzed programs’ cost growth between three 


Page 182 


GAO-19-336SP Weapon Systems Annual Assessment 




Appendix II: Objectives, Scope, and 
Methodology 


acquisition phase intervals based on the program’s knowledge points: 
development start to critical design review, critical design review to 
production decision, and from production start until end of production. We 
then calculated the total cost growth for the entire portfolio within these 
intervals. 

For the sixth observation, cost changes related to quantity changes were 
factored out of the total acquisition cost change calculation used to 
identify programs with the largest acquisition cost increase and decrease. 
We ranked the ten programs with the highest overall cost change and the 
ten programs with the lowest overall cost change both in terms of their 
total cost and as a percentage since their initial estimates. 

Our seventh observation focuses on the performance of programs 
initiated after and prior to 2010. DOD implemented selected requirements 
of the Weapon Systems Acquisition Reform Act (WSARA) on December 
4, 2009, and programs with succeeding acquisition start dates were 
classified as “post-2010” programs. We compared the cost and schedule 
changes of both groups, then demonstrated their total acquisition cost as 
a percentage, apportioned between the two groups and costs of F-35, 
DOD’s largest pre-2010 acquisition program. Furthermore, for both 
groups of programs, we analyzed cost growth between three acquisition 
phase intervals based on the program’s knowledge points: development 
start to critical design review, critical design review to production decision, 
and from production start until end of production. We then calculated the 
total cost growth for the entire portfolio within these intervals. In addition, 
we analyzed buying power performance of both groups over the past 
year. Data elements used to complete this calculation are the number of 
procurement units, procurement cost changes, and average procurement 
unit costs per program. We calculated cost change “due” to quantity 
changes as the change in quantity over the last year multiplied by the 
average procurement unit cost for the program a year ago. We calculated 
cost change “not due” to quantity changes as the current acquisition 
quantity times the change in average per unit costs. In practice, changes 
in quantity will often affect per unit cost—as discussed later in the 
appendix—so this is more precisely described as “Cost change due to 
change in quantity assuming no change in average procurement unit 
cost” and “Cost change due to changes in average procurement unit 
cost.” If changes in quantity affect per unit cost, those changes will 
appear in the cost change “not due” to quantity changes. We first 
calculated the amount of procurement cost growth attributable to quantity 
changes. To do this, we multiplied any change in quantity by the average 
procurement unit cost for the program a year ago. The resulting dollar 
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amount is considered a change due solely to shifts in the number of units 
procured and may overestimate the amount of change expected when 
quantities increase. Additionally, it could also underestimate the expected 
change when quantities decrease as it does not account for other effects 
of quantity changes on procurement such as gain or loss of learning in 
production that could result in changes to unit cost over time or the use or 
absence of economic orders of material. However, these changes are 
accounted for as part of the change in cost not due to quantities. 


Analysis of Portfolio’s 

Competitive 

Environment 


We utilized data from the Federal Procurement Data System-Next 
Generation (FPDS-NG) and DAMIR to develop the two competition- 
related observations (observations eight and nine). More specifically, we 
compiled all data related to the contracts listed on each program’s 
December 2017 SAR, then queried the FPDS-NG database to obtain the 
data used to analyze the competitive environment of the 2018 MDAP 
portfolio. By statute, SARs include an MDAP’s “major contracts,” which 
are defined as “each of the six largest prime, associate, or Government- 
furnished equipment contracts under the program that is in excess of 
$40,000,000 and that is not a firm, fixed price contract.” 1 An MDAP’s SAR 
requirements cease after 90 percent of the total quantity of items to be 
purchased under the program are delivered or 90 percent of planned 
expenditures are made. 2 Additionally, DOD guidance states that once a 
contract is 90 percent complete, it is no longer reported in the SAR. We 
added data elements from DAMIR, namely the account type, to the 
FPDS-NG dataset. 


We assessed the reliability of FPDS-NG data by performing electronic 
testing of selected data elements as well as reviewing existing information 
about the system and its data. Specifically, we reviewed the data 
dictionary, data validation rules, and the fiscal year 2017 Federal 
Government Procurement Data Quality Summary for Agency Data in the 
Federal Procurement Data System. We found the FPDS-NG data 
sufficiently reliable for the purposes of this report. 

We defined noncompetitive obligations to include obligations through 
contracts that were awarded using the exceptions to full and open 
competition listed in Federal Acquisition Regulation (FAR) Subpart 6.3 


^0 U.S.C. § 2432(a)(3), (e)(8). 
2 10 U.S.C. § 2432(g). 


Page 184 


GAO-19-336SP Weapon Systems Annual Assessment 





Appendix II: Objectives, Scope, and 
Methodology 


(Other than Full and Open Competition). We also included 
noncompetitive orders issued under multiple award indefinite 
delivery/indefinite quantity contracts or under the General Service 
Administration’s schedules program. For noncompetitive contract actions, 
we included contracts and orders coded as “not competed,” “not available 
for competition,” and “not competed under simplified acquisition 
procedures,” as well as orders coded as an exception to “subject to fair 
opportunity,” including “urgency,” “only one source,” “minimum 
guarantee,” “follow-on action following competitive initial action,” “other 
statutory authority,” and “sole source.” For competitive contract actions, 
we included contracts and orders coded as “full and open competition,” 
“full and open after exclusion of sources,” and “competed under simplified 
acquisition procedures” as well as orders subject to multiple award fair 
opportunity coded as “subject to fair opportunity” and as “fair opportunity 
given,” and “competitive set aside.” We calculated competition rates as 
the number of competitive contracts and orders over all contracts and 
orders reported in the December 2017 SARs. 

In observation eight, we used the FPDS-NG dataset to identify the 
number of offers received for each contract award, as well as the extent 
to which the obligations were for contracts or orders that were 
competitively awarded. In addition, we examined the extent to which the 
contracts and orders in the dataset were awarded competitively across 
the managing military departments and, for each, calculated the total 
value of the competitive versus non-competitive awards. In order to 
conduct this analysis, we included contract or account-type identifying 
data from DAMIR to our FPDS-NG dataset. 

For observation nine, we used FPDS-NG data elements containing parent 
company information of contract awardees and the total base and all 
options values of the awards to identify the five parent corporations with 
the highest amounts of contract awards. We used parent company 
information contained in the FPDS-NG dataset in order to group 
companies associated with the five contractors receiving the greatest 
value of awards. 


Analysis of Selected 
Acquisition Reforms 


To determine the extent to which the department has transferred 
acquisition programs’ decision authority to the military departments, we 
obtained and analyzed the annual MDAP lists from DAMIR dating back to 
2012. Each list identifies the acquisition authority for each program. For 
each year of data, we calculated both the number of programs for which 
DOD was the decision authority and the number of programs for which 
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the military department was the acquisition authority. Based on that data, 
we created a column chart that illustrates how the decision authority has 
shifted each year across the portfolio. 

To develop observations on the extent to which DOD is applying recent, 
selected acquisition reforms, we reviewed specific acquisition reform 
provisions contained in the 2016, 2017, and 2018 NDAAs. We selected 
reforms that, in our view, involve oversight and acquisition principles 
consistent with our knowledge-based practices and provide mechanisms 
for more rapid fielding of warfighter capability. We analyzed questionnaire 
data received from the 45 current and six future MDAPs in our 
assessment to determine the extent to which acquisition reforms have 
been applied or gained traction within DOD. Based on the questionnaire 
responses, we analyzed the degree to which the programs applied 
specific acquisition reform provisions contained in the 2016, 2017, and 
2018 NDAAs. Those provisions are summarized separately in appendix 
IV. 


Analysis of Selected 
DOD Programs Using 
Knowledge-Based 
Criteria and Related 
Implications for 
Testing and Software 
Development 


To collect data from current and future MDAPs—including cost and 
schedule estimates, technology maturity, and planned implementation of 
acquisition reforms—we distributed two electronic questionnaires—one 
questionnaire for the 45 current programs and a slightly different 
questionnaire for the six future programs. Both of the questionnaires were 
web-based so that respondents could respond and submit their answers 
online. We received responses from all of the programs we assessed 
from October 2018 to January 2019. To ensure the reliability of the data 
collected through our questionnaires, we took a number of steps to 
reduce measurement error and non-response error. 

These steps included conducting two pretests of the future MDAP 
questionnaire and three pretests for the current major defense acquisition 
program questionnaire prior to distribution to ensure that our questions 
were clear, unbiased, and consistently interpreted. Our pretests covered 
each branch of the military to better ensure that the questionnaires could 
be understood by officials within each branch. We determined that the 
data from the SARs and DAMIR were sufficiently reliable for the purposes 
of this report. 


Our analysis of how well programs are adhering to a knowledge-based 
acquisition approach focuses on 45 MDAPs that mare mostly in 
development or the early stages of production. To assess the knowledge 
attained by key decision points (system development start or detail design 
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contract award for shipbuilding programs, critical design review or lead 
ship fabrication start for shipbuilding programs, and production start), we 
collected data from program offices about their knowledge at each point. 

We also provide information on how much knowledge was obtained at 
key decision points by programs that accomplished these previously. We 
also include observations on the knowledge that the six future programs 
expect to obtain before starting development. We did not validate the data 
provided by the program offices, but reviewed the data and performed 
various checks to determine that they were reliable for our purposes. 
Where we discovered discrepancies, we clarified the data accordingly. 

For programs that have passed a key decision point and have since been 
restructured, we assessed them against their original cost and schedule 
estimates at that milestone or decision point, such as development start. 
We did not reassess programs at milestones they had previously 
reached, as in cases where a program is repeating a key decision point or 
milestone, such as milestone B. We keep our original assessment of the 
program’s knowledge attained at the original milestone. However, we do 
change future milestone dates if those milestone had yet to be reached, 
and we assess those programs for their implementation of our best 
practices at that point in time. 

For the second consecutive year, we performed an exploratory statistical 
analysis that examined our identified knowledge-based acquisition 
practices and select programs’ cost and schedule changes. We focused 
the analysis on the 17 non-shipbuilding MDAPs that, prior to this 
assessment, completed each of the three knowledge points within the 
acquisition process (i.e., completed development, held a critical design 
review, and started production). Our statistical analysis compared 
average cost and schedule changes for those programs that had 
implemented eight key knowledge-based acquisition practices by the time 
they reached knowledge points 1 through 3, compared to those programs 
that did not complete the best practices at each knowledge point. To 
ensure a minimally reliable estimate of the average in each group, we 
limited our analysis to those knowledge-based acquisition practices for 
which at least three programs had engaged in the practice, and at least 
three programs had not engaged in the practice. Although we sought to 
assess the statistical significance of demonstrating technologies to form, 
fit, and function within a realistic environment, we observed that only one 
program in the sample demonstrated this level of technology maturity 
before it started development. This one program provided an insufficient 
basis to determine whether this best practice corresponded with lower 


Page 187 


GAO-19-336SP Weapon Systems Annual Assessment 




Appendix II: Objectives, Scope, and 
Methodology 


cost and schedule growth. We assessed the statistical significance of the 
observed differences between the groups at the 90 percent confidence 
level. 3 With such a small sample of MDAPs, our estimates are fairly 
imprecise and do not meet normality assumptions. 

To assess programs’ developmental testing and production concurrency, 
we identified the programs—among those we included in our 
assessment—with production start dates. We used the questionnaire 
responses from those programs to identify the dates for the start and end 
of developmental testing, compared those dates to the timing of each 
program’s production decision, and determined whether production- 
representative prototype testing would conclude before production start. 

To examine programs’ software development efforts, we identified the 
dates reported by programs for their software and hardware integration 
and compared those dates to each program’s production start date to 
assess each program’s degree of software development and production 
concurrency. 

To examine when programs were declaring initial operational capability 
compared to finishing their operational testing, we identified programs’ 
reported initial operational capability dates and start and end dates of 
their operational testing. We determined whether the initial operational 
capability date was before, during or after its testing dates. Based on our 
determination, we summed and analyzed what percent of programs were 
in each category. For some programs, either one or both of these dates 
were not available. 

The 53 current and future programs included in our assessment were in 
various stages of the acquisition cycle, and not all of the programs 
provided information on knowledge obtained at each point. Programs 
were not included in our assessments at key decision points if relevant 
data were not available. Our analysis of knowledge attained at each key 
point includes factors that we have previously identified as underpinning a 
knowledge-based acquisition approach, including holding early systems 
engineering reviews, testing an integrated prototype prior to the design 
review, using a reliability growth curve, planning for manufacturing, and 
testing a production-representative prototype prior to making a production 


Statistical significance at the 90 percent confidence level indicates that the chances of 
observing a statistical difference as large or larger as observed by chance, if no difference 
existed, is less than 10 percent. 
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decision. Additional information on how we collect these data is found in 
the production knowledge assessment section of this appendix. See also 
appendix III for a list of the practices that are associated with a 
knowledge-based acquisition approach. 


Individual Assessments Of This report presents individual assessments of 51 current and future 
Weapon Programs MDAPs. a table listing these assessments is found in appendix X. Of our 

51 total assessments, 39 are captured in a two-page format discussing 
technology, design, and manufacturing knowledge obtained and other 
program issues. These two-page assessments are of current MDAPs, 
most of which are in development or early production. The remaining 12 
assessments we presented in a one-page format that describes their 
current status. Those one-page assessments include (1) six future 
MDAPs; (2) one MDAP that is in development, but released a baseline 
late in our review—VC-25B Presidential Aircraft Recapitalization, and (3) 
five MDAPs that are well into production, but planning to introduce new 
increments of capability—specifically, the Navy’s DDG 51 Arleigh Burke 
Class Destroyer, Flight III; LHA 8 America Class Amphibious Assault 
Ship; LPD 17 San Antonio Class Amphibious Transport Dock, Flight II; P- 
8A Poseidon, Increment 3; and SSN 774 Virginia Class Submarine, Block 
V. We collected data on these programs through January 31, 2019, with 
the exception of one program—Offensive Anti-Surface Warfare Increment 
1—which released a new baseline in early February 2019 that we 
included. 

For presentation purposes we grouped the individual assessments by 
lead military department—Army, Navy and Marine Corps, Air Force, and 
DOD-wide—and inserted a summary analysis page at the start of each 
grouping. These four summary analysis pages present aggregated 
information about selected programs’ acquisition phases, funding needs, 
knowledge attainment, cost and schedule performance, oversight 
authorities, software characteristics, and competition in contracting. We 
obtained this data primarily from December 2017 SARs and 
supplemented by program office responses to our questionnaire. We 
reported cost and schedule growth in the summary analysis pages in a 
manner that is consistent with how it is reported and described elsewhere 
in the report. Of note, the cost analysis in the summary analysis pages is 
based on cost data reported in the December 2017 SARs. These data 
may differ from the estimates reported in the individual assessments, 
which in some cases are more recent than December 2017. 
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Over the past several years, DOD has revised policies governing weapon 
system acquisitions and changed the terminology used for major 
acquisition events. To make DOD’s acquisition terminology more 
consistent across our individual program assessments, we standardized 
the terminology for key program events. For most individual programs in 
our assessment, “development start” refers to the initiation of an 
acquisition program as well as the start of either engineering and 
manufacturing development or system development. This generally 
coincides with DOD’s milestone B. A few programs in our assessment 
have a separate “program start” date, which begins a pre-system 
development phase for program definition and risk-reduction activities. 
This “program start” date generally coincides with DOD’s former 
terminology for milestone I or DOD’s current milestone A, which denotes 
the start of technology maturation and risk reduction. The “production 
decision” generally refers to the decision to enter the production and 
deployment phase, typically with low-rate initial production. The “initial 
capability” refers to the initial operational capability—sometimes called 
first unit equipped or required asset availability. For shipbuilding 
programs, the schedule of key program events in relation to acquisition 
milestones varies for each program. Our work on shipbuilding best 
practices has identified the detail design contract award and the start of 
lead ship fabrication as the points in the acquisition process roughly 
equivalent to development start and design review for other programs. 

We obtained the information presented in the “Program Essentials” 
section of the individual assessments from the DAMIR system, program 
office responses to a questionnaire, program office documents, and 
communications with program officials. As a result, DOD is the source of 
the information regarding the milestone decision authority, identity of the 
contractors, the contract types, and the software development 
approach. We did not review individual contracts for each system. In their 
questionnaire responses, program offices self-identified the type of 
software development approach used based on definitions from the 
Defense Acquisition University. In cases where program offices identified 
more than one type, we describe this as a “mixed” approach. 

For each program we assessed in a two-page format, we present cost, 
schedule, and quantity data at the program’s first full estimate as well as 
an estimate from the latest SAR or the most recent Defense Acquisition 
Executive Summary (DAES) report as of June or July 2018, except in 
cases where significant changes subsequently occurred, such as a new 
program baseline. For cases in which we received updated schedule data 
after the date of the DAES reports, we followed up with programs to 
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determine whether there were any cost changes associated with the 
schedule changes and updated the data as needed. The first full estimate 
is generally the cost estimate established at milestone B—development 
start; however, for a few programs that did not have such an estimate, we 
used the estimate at milestone C—production start—instead. For 
shipbuilding programs, we used their planning estimates if those 
estimates were available. For systems for which a first full estimate was 
not available, we only present the latest available estimate of cost and 
quantities. For the other programs assessed in a one-page format, we 
present the latest available estimate of cost and quantity from the 
program office. 

For each program we assessed, all cost information is presented in fiscal 
year 2019 dollars. We converted cost information to fiscal year 2019 
dollars using conversion factors from the DOD Comptroller’s National 
Defense Budget Estimates for Fiscal Year 2019 (table 5-9). We have 
depicted only the program’s main elements of acquisition cost—research 
and development and procurement. However, the total program cost also 
includes military construction and acquisition-related operation and 
maintenance costs. Because of rounding and these additional costs, in 
some situations, total cost may not match the exact sum of the research 
and development and procurement costs. The program unit costs are 
calculated by dividing the total program cost by the total quantities 
planned. These costs are often referred to as program acquisition unit 
costs. In some instances, the data were not applicable, and we annotate 
this by using the term “not applicable (NA).” The quantities listed refer to 
total quantities, including both procurement and development quantities. 

The schedule assessment for each program is based on acquisition cycle 
time, defined as the number of months between program start and the 
achievement of initial operational capability or an equivalent fielding date. 
In some instances the data were not yet available, and we annotate this 
by using the term “to be determined (TBD)” or “NA.” As a result of 
updates to our methods for calculating acquisition cycle time for the 2019 
assessment, some first full estimates may reflect a slight adjustment (e.g., 
1 month) from what we reported in previous years. 

The information presented in “Funding and Quantities” is from fiscal year 
2019 through completion and draws on information from the latest SAR 
(for quantities in particular) or the most recent DAES report as of June or 
July 2018 reflecting 2019 data, except in cases where significant changes 
subsequently occurred, such as a new program baseline. The quantities 
listed refer only to procurement quantities. Satellite programs, in 
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particular, produce a large percentage of their total operational units as 
development quantities, which are not included in the quantity figure. 

The intent of these comparisons is to provide an aggregate, or overall, 
picture of a program’s history. These assessments represent the sum of 
the federal government’s actions on a program, not just those of the 
program manager and the contractor. DOD does a number of detailed 
analyses of changes that attempt to link specific changes with triggering 
events or causes. Our analysis does not attempt to make such detailed 
distinctions. 

We also reviewed whether individual subcontracting reports from a 
program’s prime contractor or contractors were accepted on the 
Electronic Subcontracting Reporting System (eSRS). We reviewed this 
information for 82 of the MDAPs in the 2018 portfolio using the contract 
information reported in their December 2017 SARs. See appendix I for a 
list of the programs we reviewed. 


Product Knowledge Data In our past work examining weapon acquisition issues and knowledge- 
on Individual Two-Page based acquisition practices for product development, we have found that 
Accaccmontc leading commercial firms pursue an acquisition approach that is anchored 

in knowledge, whereby high levels of product knowledge are 
demonstrated by critical points in the acquisition process. On the basis of 
this work, we have identified three key knowledge points during the 
acquisition cycle—system development start, critical design review, and 
production start—at which programs need to demonstrate critical levels of 
knowledge to proceed. To assess the product development knowledge of 
each program at these key points, we reviewed questionnaires submitted 
by programs; however, not every program had responses to each 
element of the questionnaire. We also reviewed pertinent program 
documentation and discussed the information presented on the 
questionnaire with program officials as necessary. 

For our “Attainment of Product Knowledge” tables, we assessed the 
programs’ current status in implementing the knowledge-based 
acquisition practices criteria, as well as the programs’ progress in meeting 
the criteria at the time they reached the three key knowledge points 
during the acquisition cycle—system development start, critical design 
review, and production start. For programs that have passed a key 
decision point and have since been restructured, we continue to assess 
them against their original cost and schedule estimates at that milestone 
or decision point, such as development start. We have not reassessed a 
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program at milestones that have already been reached if a program is 
repeating a key decision point or milestone, such as milestone B. We 
have kept our original assessment of the program’s knowledge attained at 
the original milestone. However, we have changed future milestone dates 
in instances when the program had not yet reached the affected 
milestone. In these instances, we assessed the program for its 
implementation of our knowledge-based acquisition practices criteria at 
that point in time. To assess a program’s readiness to enter system 
development, we collected data through the questionnaire on critical 
technologies and early design reviews. To assess technology maturity, 
we asked program officials to apply a tool, referred to as technology 
readiness levels (TRL), for our analysis. The National Aeronautics and 
Space Administration originally developed TRLs, and the Army and Air 
Force science and technology research organizations use them to 
determine when technologies are ready to be handed off from science 
and technology managers to product developers. TRLs are measured on 
a scale from 1 to 9, beginning with paper studies of a technology’s 
feasibility and culminating with a technology fully integrated into a 
completed product. See appendix VIII for TRL definitions. Our knowledge- 
based acquisition practices work has shown that a TRL 7—demonstration 
of a technology in its form, fit, and function within a realistic 
environment—is the level of technology maturity that constitutes a low 
risk for starting a product development program. 4 For shipbuilding 
programs, we have recommended that this level of maturity be achieved 
by the contract award for detail design. 5 In our assessment, the 
technologies that have reached TRL 7, a prototype demonstrated in a 
realistic environment, are referred to as mature or fully mature. Those 
technologies that have reached TRL 6, a prototype very close to final 
form, fit, and function demonstrated within a relevant environment, are 
referred to as approaching or nearing maturity. Satellite technologies that 
have achieved TRL 6 are assessed as fully mature due to the difficulty of 
demonstrating maturity in a realistic environment—space. In addition, we 
asked program officials to provide the date of the system-level preliminary 
design review. We compared this date to the system development start 
date. 


4 GAO, Best Practices: Better Management of Technology Development Can Improve 
Weapon System Outcomes, GAO/NSIAD-99-162 (Washington, D.C.: July 30, 1999); 
GAO, Best Practices: Better Matching of Needs and Resources Will Lead to Better 
Weapon System Outcomes, GAO-01-288 (Washington, D.C.: Mar. 8, 2001). 

5 GAO, Best Practices: High Levels of Knowledge at Key Points Differentiate Commercial 
Shipbuilding from Navy Shipbuilding, GAO-09-322 (Washington, D.C.: May 13, 2009). 
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In most cases, we did not validate the program offices’ selection of critical 
technologies or the determination of the demonstrated level of maturity. 
We sought to clarify the TRLs in those cases where information existed 
that raised questions. If we were to conduct a detailed review, we may or 
may not adjust the critical technologies assessed, their readiness levels 
demonstrated, or both. It was not always possible to reconstruct the 
technological maturity of a weapon system at key decision points after the 
passage of many years. Where practicable, we compared technology 
assessments provided by the program office to assessments by officials 
from the Office of the Assistant Secretary of Defense for Research and 
Engineering. 

To assess design stability, we asked program officials to provide the 
percentage of design drawings completed or projected for completion by 
the design review, the production decision, and as of our current 
assessment in the questionnaire. In most cases, we did not verify or 
validate the percentage of engineering drawings provided by the program 
office. We clarified the percentage of drawings completed in cases where 
information raised questions. Completed drawings were defined as the 
number of drawings released or deemed releasable to manufacturing that 
can be considered the “build to” drawings. For shipbuilding programs, we 
asked program officials to provide the percentage of the three- 
dimensional product model that had been completed by the start of lead 
ship fabrication, and as of our current assessment. To gain greater 
insights into design stability, we also asked program officials to provide 
the date they planned to first integrate and test all key subsystems and 
components into a system-level integrated prototype. We compared this 
date to the date of the design review. We did not assess whether 
shipbuilding programs had completed integrated prototypes. 

To assess production maturity, we asked program officials for their 
Manufacturing Readiness Level (MRL) for process capability and control 
or to identify the number of critical manufacturing processes and, where 
available, to quantify the extent of statistical control achieved for those 
processes as a part of our questionnaire. In most cases, we did not verify 
or validate the information provided by the program office. We clarified 
the number of critical manufacturing processes and the percentage of 
statistical process control where information existed that raised questions. 
We used a standard called the Process Capability Index, a process- 
performance measurement that quantifies how closely a process is 
running to its specification limits. The index can be translated into an 
expected product defect rate, and we have found it to be a best practice. 
We also used data provided by the program offices on their MRL for 
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process capability and control, a sub-thread tracked as part of the 
manufacturing readiness assessment process recommended by DOD, to 
determine production maturity. We assessed programs as having mature 
manufacturing processes if they reported an MRL 9 for that sub-thread— 
meaning that manufacturing processes are stable, adequately controlled, 
and capable. To gain further insights into production maturity, we asked 
program officials whether the program planned to demonstrate critical 
manufacturing processes on a pilot production line before beginning low- 
rate initial production. We also asked programs on what date they 
planned to begin system-level developmental testing of a fully configured, 
production-representative prototype in its intended environment. We 
compared this date to the production start date. We did not assess 
production maturity for shipbuilding programs. 

Although the knowledge points provide indicators of potential risks, by 
themselves they do not cover all elements of risk that a program 
encounters during development, such as funding instability. 


Technology Maturation 
Knowledge Data on 
Individual One-Page 
Assessments 


For future MDAPs in this year’s assessment, we included a table, 
“Attainment of Technology Maturation Knowledge,” indicating whether the 
programs had attained or planned to attain key knowledge prior to starting 
development. We selected key activities programs should conduct prior to 
entering system development, based on DOD’s Instruction 5000.02: 
conduct competitive prototyping, validate requirements, complete a 
technology readiness assessment, and complete a preliminary design 
review. These are not the only activities contemplated at this stage, but 
the table is intended to provide insight into the extent to which a program 
has gained critical knowledge before milestone B. To determine whether 
programs had conducted or planned to conduct these activities, we 
obtained information through our questionnaire and clarified responses 
with program officials, as needed. 

We conducted this performance audit from May 2018 to May 2019, in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 
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Our prior work on best product development practices found that 
successful programs take steps to gather knowledge that confirms their 
technologies are mature, their designs stable, and that their production 
processes are in control. Successful product developers ensure a high 
level of knowledge is achieved at key junctures in development. We 
characterize these junctures as knowledge points. The Related GAO 
Products section of this report includes references to the body of work 
that helped us identify these practices and apply them as criteria in 
weapon system reviews. Table 19 summarizes these knowledge points 
and associated practices. 


Table 19: Best Practices for Knowledge-Based Acquisitions 


Knowledge Point 1: Technologies, time, funding, and other resources match customer needs. Decision to invest in product 
development 

Demonstrate technologies to a high readiness level—Technology Readiness Level 7—to ensure technologies are fit, form, function, 
and work within a realistic environment 3 

Ensure that requirements for product increment are informed by system-level preliminary design review using system engineering 
process (such as prototyping of preliminary design) 

Establish cost and schedule estimates for product on the basis of knowledge from system-level preliminary design using system 
engineering tools (such as prototyping of preliminary design) 

Constrain development phase (5 to 6 years or less) for incremental development 

Ensure development phase fully funded (programmed in anticipation of milestone) 

Align program manager tenure to complete development phase 

Contract strategy that separates system integration and system demonstration activities 

Conduct independent cost estimate 

Conduct independent program assessment 

Conduct major milestone decision review for development start 

Knowledge Point 2: Design is stable and performs as expected. Decision to start building and testing production- 
representative prototypes. 

Complete system critical design review 

Complete 90 percent of engineering design drawing packages 

Complete subsystem and system design reviews 

Demonstrate with system-level integrated prototype that design meets requirements 
Complete failure modes and effects analysis 
Identify key system characteristics 
Identify critical manufacturing processes 

Establish reliability targets and growth plan on the basis of demonstrated reliability rates of components and subsystems 
Conduct independent cost estimate 
Conduct independent program assessment 

Conduct major milestone decision review to enter system demonstration 
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Knowledge Point 3: Production meets cost, schedule, and quality targets. Decision to produce first units for customer. 

Demonstrate manufacturing processes on a pilot production line 

Build and test production-representative prototypes to demonstrate product in intended environment 
Test production-representative prototypes to achieve reliability goal 
Collect statistical process control data 

Demonstrate that critical processes are capable and in statistical control 

Conduct independent cost estimate 

Conduct independent program assessment 

Conduct major milestone decision review to begin production 

Source: GAO | GAO-19-336SP 

department of Defense policy permits development to start at a technology maturity level 
commensurate with Technology Readiness Level 6—demonstration of program technology in a 
relevant environment. Therefore we have assessed programs against this measure as well. 
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In the National Defense Authorization Acts (NDAA) for Fiscal Years 2016, 
2017, and 2018, Congress introduced numerous acquisition reforms that 
pertain to the performance of weapon acquisition programs and the 
speed at which the Department of Defense (DOD) delivers capabilities to 
the warfighter. For this assessment, we reviewed major defense 
acquisition programs (MDAP) application of six of these reforms. 1 

Section 864 of the 2018 NDAA increases the dollar thresholds at which 
approval by the service acquisition executive or Under Secretary for 
Acquisition, Technology and Logistics is required for use of other 
transaction authority for prototype projects. Specifically, the threshold for 
service acquisition executive approval was raised from $50 million to 
$100 million, and the threshold for Under Secretary of Acquisition, 
Technology and Logistics approval was raised from $250 million to $500 
million. 2 

Section 815 of the 2016 NDAA codified DOD’s other transaction 
authority for prototype projects at 10 U.S.C. § 2371b. This section, as 
amended, stipulated that to use the authority, one of the following 
conditions must be met: 

• There is at least one nontraditional defense contractor or nonprofit 
research institution participating to a significant extent in the prototype 
project. 3 

• All significant participants, other than the federal government, must be 
small businesses (including small businesses participating in a 
program described under section 9 of the Small Business Act (15 
U.S.C. § 638)) or nontraditional defense contractors. 


1 Although we list seven reforms in this appendix, we asked programs about actions they 
took related to only six of them. We included Section 815 of the 2016 NDAA in this 
appendix as it provides additional context regarding conditions that must be met for DOD’s 
utilization of Other Transaction Authority for prototype projects. 

2 Section 211 of the fiscal year 2019 NDAA replaced the Under Secretary of Defense for 
Acquisition, Technology and Logistics as the approval authority with the Under Secretary 
of Defense for Research and Engineering or the Under Secretary of Defense for 
Acquisition and Sustainment. Pub. L. No. 115-232, § 211 (2018). 

3 Nontraditional defense contractor means an entity that is not currently performing and 
has not performed any contract or subcontract for DOD that is subject to full coverage 
under the cost accounting standards prescribed pursuant to 41 U.S.C. § 1502 and the 
regulations implementing such section, for at least the 1-year period preceding the 
solicitation of sources by DOD for the procurement. 10 U.S.C. § 2302(9). Section 815 is 
codified, as amended, at 10 U.S.C. § 2371b. 
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• At least one third of the total cost of the prototype project is paid out of 
funds provided by sources other than the federal government. 

• The senior procurement executive for the agency determines in 
writing that exceptional circumstances justify the use of a transaction 
that provides for innovative business arrangements or structures that 
would not be feasible or appropriate under a contract, or would 
provide an opportunity to expand the defense supply base in a 
manner that would not be practical or feasible under a contract. 


Section 807 of the 2017 NDAA established that before funds are 
obligated for the technology development, systems development, or 
production of an MDAP, DOD must establish goals that include: 

• the procurement unit cost and sustainment cost 

• the date for initial operational capability, and 

• technology maturation, prototyping, and a modular open system 
approach to evolve system capabilities and improve interoperability. 


This section also requires an independent technical risk assessment 
before any decision is made to grant Milestone A approval for a program 
that identifies critical technologies and manufacturing processes that 
need to be matured. The assessment is also required prior to any 
decisions to grant Milestone B approval, enter low-rate production or full- 
rate production, or at any other time deemed appropriate by the Secretary 
of Defense, for a program that identifies any critical technologies or 
manufacturing processes that have not been successfully demonstrated 
in a relevant environment. 

Section 805 of the 2017 NDAA provides that a MDAP that receives 
Milestone A or Milestone B approval after January 1, 2019 must be 
designed and developed, to the maximum extent practicable, with a 
modular open system approach to enable incremental development and 
enhance competition, innovation, and interoperability. 4 DOD’s Better 


4 The Department of Defense’s modular open systems approach (MOSA) is to design 
systems with highly cohesive, loosely coupled, and severable modules that can be 
competed separately and acquired from independent vendors. This approach allows the 
department to acquire warfighting capabilities, including systems, subsystems, software 
components, and services, with more flexibility and competition. MOSA implies the use of 
modular open systems architecture, a structure in which system interfaces share common, 
widely accepted standards, with which conformance can be verified. 
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Buying Power initiatives have emphasized the use of modular open 
systems architecture. 

Section 804 of the 2016 NDAA directs DOD to establish guidance for a 
“middle tier” of acquisition programs that are intended to be completed in 
a period of two to five years. The guidance was required to establish two 
acquisition pathways: 

• Rapid prototyping: provides for the use of innovative technologies to 
rapidly develop fieldable prototypes to demonstrate new capabilities to 
meet emerging military needs. The objective of an acquisition 
program under this pathway is to field a prototype that can be 
demonstrated in an operational environment and provide for a 
residual operational capability within five years of the development of 
an approved requirement. 

• Rapid fielding: provides for the use of proven technologies to field 
production quantities of new or upgraded systems with minimal 
development required. The objective of an acquisition program under 
this pathway shall be to begin production within six months and 
complete fielding within five years of the development of an approved 
requirement. 


Section 806 of the 2016 NDAA authorizes the Secretary of Defense to 
waive any provision of acquisition law or regulation addressing the 
establishment of a specification for the capability to be acquired; 
research, development, test, and evaluation of the capability to be 
acquired; production, fielding, and sustainment of the capability to be 
acquired; and solicitation, selection of sources, and award of contracts for 
the capability to be acquired. The Secretary may issue such waivers after 
determining that the acquisition of the capability is in the vital national 
security interests of the United States, the application of the law or 
regulation to be waived would impede the acquisition of the capability in a 
manner that would undermine the national security of the United States, 
and the underlying purpose of the law or regulation to be waived can be 
addressed in a different manner or at a different time. 

Section 823 of the 2016 NDAA amends 10 U.S.C. § 2366a—the statute 
that governs Milestone A approval—to require the milestone decision 
authority to provide a written determination—as opposed to the previously 
required certification—that the program meets certain conditions before 
granting Milestone A approval. Some of the conditions were carried over 
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from the prior version of the statute, while others were new. The 

conditions are summarized as follows: 

• The program fulfills an approved Initial Capabilities Document; 

• The program was developed after appropriate market research; 

• If the program duplicates a capability already provided by an existing 
system, the duplication is necessary and appropriate; 

• A plan is in place to reduce any identified areas of risk; 

• A plan for sustainment has been addressed and a determination of 
applicability of core logistics capabilities requirements has been 
made; 

• An analysis of alternatives has been performed consistent with the 
study guidance developed by the Director of Cost Assessment and 
Program Evaluation; and 

• A cost estimate for the program has been submitted, with the 
concurrence of the Director of Cost Assessment and Program 
Evaluation, and the level of resources required to develop, procure, 
and sustain the program is sufficient for successful program execution 
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Table 20 shows total acquisition cost amounts from first full estimates and 
from December 2016 and 2017 select acquisition reports for each 
program in the Department of Defense’s (DOD) 2018 portfolio of major 
weapon programs. Programs are organized by their managing military 
department and changes in total acquisition cost since last year and since 
first full estimates are displayed as percentages. 


Table 20: December 2017 Cost Estimates and First Full Estimates for the Department of Defense’s 2018 Portfolio of Major 
Defense Acquisition Programs 

Fiscal year 2019 dollars (in millions) 

Air Force 


Program name 

First full 
estimate of 
total 
acquisition 
cost 

December 

2016 

estimated 

total 

acquisition 

cost 

December 

2017 

estimated 

total 

acquisition 

cost 

Percentage 
change in 
total 
acquisition 
cost since 
December 
2016 

Percentage 
change in 
total 
acquisition 
cost since 
first full 
estimate 

Advanced Extremely High Frequency (AEHF) Satellite 

3,047.97 

2,820.02 

2,810.43 

-0.3 

-7.8 

AIM-120 Advanced Medium Range Air-to-Air Missile 
(AMRAAM) 

12,237.68 

25,478.53 

25,778.97 

1.2 

110.7 

Airborne Warning and Control System Block 40/45 

Upgrade (AWACS Blk 40/45 Upgrade) 

3,126.27 

3,004.21 

3,077.76 

2.4 

-1.6 

B-2 Defensive Management System Modernization (B-2 
DMS-M) 

2,686.21 

2,666.56 

2,757.46 

3.4 

2.7 

B61 Mod 12 Life Extension Program Tailkit Assembly 
(B61 Mod 12 LEPTKA) 

1,463.89 

1,262.30 

1,246.23 

-1.3 

-14.9 

C-130J Hercules Transport Aircraft (C-130J) 

1,062.53 

17,704.67 

16,353.50 

-7.6 

1439.1 

Combat Rescue Helicopter (CRH) 

8,698.96 

8,852.17 

8,365.77 

-5.5 

-3.8 

Evolved Expendable Launch Vehicle (EELV) 

19,455.06 

58,588.83 

57,036.66 

-2.6 

193.2 

F-15 Eagle Passive/Active Warning and Survivablity 
System (F-15 EPAWSS) 

4,467.74 

2,712.69 

2,728.35 

0.6 

-38.9 

F-22 Increment 3.2B Modernization (F-22 Inc 3.2B Mod) 

1,677.87 

1,549.08 

1,556.64 

0.5 

-7.2 

Family of Advanced Beyond Line-of-Sight Terminals 
(FAB-T) 

3,570.19 

4,716.69 

4,705.63 

-0.2 

31.8 

Global Positioning System III (GPS III) 

4,413.46 

5,841.52 

5,841.86 

0.0 

32.4 

HC/MC-130 Recapitalization Aircraft (HC/MC-130 Recap) 

9,364.83 

14,591.47 

14,660.45 

0.5 

56.5 

Intercontinental Ballistic Missile Fuze Modernization 
(ICBM Fuze Mod) 

1,951.19 

1,949.68 

1,995.48 

2.3 

2.3 

Joint Air-to-Surface Standoff Missile Extended Range 
(JASSM-ER) 

2,524.44 

4,413.57 

4,515.70 

2.3 

78.9 

Joint Direct Attack Munition (JDAM) 

3,828.09 

11,467.07 

12,627.86 

10.1 

229.9 
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Program name 

First full 
estimate of 
total 
acquisition 
cost 

December 

2016 

estimated 

total 

acquisition 

cost 

December 

2017 

estimated 

total 

acquisition 

cost 

Percentage 
change in 
total 
acquisition 
cost since 
December 
2016 

Percentage 
change in 
total 
acquisition 
cost since 
first full 
estimate 

KC-46 Tanker Modernization Program (KC-46A) 

49,067.76 

41,669.40 

41,215.85 

-1.1 

-16.0 

Military GPS User Equipment (MGUE) Increment 1 

1,555.80 

1,194.15 

1,460.63 

22.3 

-6.1 

MQ-9 Reaper Unmanned Aircraft System (MQ-9 Reaper) 

2,952.64 

13,199.46 

13,540.01 

2.6 

358.6 

Next Generation Operational Control System (GPS OCX) 

3,707.58 

5,636.58 

6,277.58 

11.4 

69.3 

Small Diameter Bomb Increment II (SDB II) 

5,264.52 

4,443.05 

4,518.77 

1.7 

-14.2 

Space Based Infrared System High (SBIRS High) 

3,977.45 

3,505.93 

3,507.12 

0.0 

-11.8 

Space Fence Ground-Based Radar System Increment 1 

1,684.77 

1,589.08 

1,588.65 

0.0 

-5.7 

Army 

Program name 

First full 
estimate of 
total 

acquisition 

cost 

December 

2016 

estimated 

total 

acquisition 

cost 

December 

2017 

estimated total 

acquisition 

cost 

Percentage 
change in 
total 

acquisition 
cost since 
December 
2016 

Percentage 
change in 
total 

acquisition 
cost since 
first full 
estimate 

AH-64E Apache New Build (AH-64E New Build) 

2,653.25 

1,965.61 

1,959.75 

-0.3% 

-26.1% 

AH-64E Apache Remanufacture (AH-64E 

Remanufacture) 

8,109.14 

14,262.22 

14,498.81 

1.7% 

78.8% 

Airborne and Maritime/Fixed Station (AMF) 

9,129.91 

3,360.91 

3,437.75 

2.3% 

-62.3% 

Armored Multi-Purpose Vehicle (AMPV) 

11,405.72 

11,489.31 

11,607.15 

1.0% 

1.8% 

CH-47F Improved Cargo Helicopter (CH-47F) 

3,605.14 

16,385.69 

16,389.77 

0.0% 

354.6% 

CH-47F Modernized Cargo Helicopter (CH-47F Block II) 

16,759.45 

— 

16,764.21 

0.0% 

0.0% 

Common Infrared Countermeasure (CIRCM) 

2,698.18 

2,723.28 

2,699.78 

-0.9% 

0.1% 

Guided Multiple Launch Rocket System/Guided Multiple 
Launch Rocket Sys Alt Warhead (GMLRS/GMLRS AW) 

1,980.00 

7,827.56 

15,502.47 

98.0% 

683.0% 

Integrated Air and Missile Defense (IAMD) 

5,630.19 

7,229.31 

7,249.72 

0.3% 

28.8% 

Joint Air-to-Ground Missile (JAGM) 

6,029.88 

6,022.65 

6,168.67 

2.4% 

2.3% 

Joint Light Tactical Vehicle (JLTV) 

25,232.83 

21,626.50 

24,058.17 

11.2% 

-4.7% 

Joint Tactical Radio System Handheld, Manpack, and 
Small Form Fit Radios (JTRS HMS) 

11,239.51 

9,756.12 

9,510.22 

-2.5% 

-15.4% 

M109A7 Family of Vehicles (M109A7 FOV) 

7,487.88 

7,584.57 

7,726.43 

1.9% 

3.2% 

M88A2 Heavy Equipment Recovery Combat Utility Lift 
Evacuation System (M88A2 Hercules) 

3,370.14 

3,320.29 

3,397.43 

2.3% 

0.8% 

MQ-1C Gray Eagle Unmanned Aircraft System (MQ-1C 
Gray Eagle) 

1,136.80 

6,137.44 

6,336.29 

3.2% 

457.4% 
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Patriot Advanced Capability-3 Missile Segment 
Enhancement (PAC-3 MSE) 

8,159.63 

6,965.27 

8,924.85 

28.1% 

9.4% 

UH-60M Black Hawk Helicopter (UH-60M Black Hawk) 

14,523.29 

28,348.81 

27,996.04 

-1.2% 

92.8% 

Warfighter Information Network-Tactical Increment 2 
(WIN-T Inc 2) 

4,152.07 

11,337.44 

4,791.84 

-57.7% 

15.4% 


Navy 


Program name 

First full 
estimate of 
total 
acquisition 
cost 

December 

2016 

estimated 

total 

acquisition 

cost 

December 

2017 

estimated 

total 

acquisition 

cost 

Percentage 
change in 
total 
acquisition 
cost since 
December 
2016 

Percentage 
change in 
total 
acquisition 
cost since 
first full 
estimate 

Advanced Arresting Gear (AAG) 

2,301.92 

2,115.91 

2,319.49 

9.6 

0.8 

AGM-88E Advanced Anti-Radiation Guided Missile 
(AGM-88E AARGM) 

1,792.37 

2,817.66 

2,816.32 

0.0 

57.1 

AIM-9X Block II Sidewinder (AIM-9X Blk II) 

4,473.22 

3,493.29 

3,601.53 

3.1 

-19.5 

Air and Missile Defense Radar (AMDR) 

6,258.95 

5,776.41 

5,552.92 

-3.9 

-11.3 

Amphibious Combat Vehicle (ACV) 

1,964.20 

1,861.09 

1,845.39 

-0.8 

-6.0 

CH-53K Heavy Lift Replacement Helicopter (CH-53K) 

18,538.42 

28,687.44 

28,845.42 

0.6 

55.6 

Columbia Class Ballistic Missile Submarine (SSBN 826) 

103,556.42 

102,596.09 

103,164.08 

0.6 

-0.4 

Cooperative Engagement Capability (CEC) 

3,295.61 

6,533.58 

6,795.16 

4.0 

106.2 

DDG 1000 Zumwalt Class Destroyer (DDG 1000) 

38,964.66 

24,786.55 

25,415.67 

2.5 

-34.8 

DDG 51 Arleigh Burke Class Guided Missile Destroyer 
(DDG 51) 

17,005.49 

129,791.36 

137,962.48 

6.3 

711.3 

E-2D Advanced Hawkeye Aircraft (E-2D AHE) 

16,518.04 

22,734.87 

22,320.77 

-1.8 

35.1 

EA-18G Growler Aircraft (EA-18G) 

10,050.63 

16,982.42 

17,017.58 

0.2 

69.3 

Fleet Replenishment Oiler (T-AO 205 Class) 

8,978.04 

— 

9,024.06 

0.5 

0.5 

Gerald R. Ford Class Nuclear Aircraft Carrier (CVN 78) 

39,835.11 

40,140.46 

49,107.01 

22.3 

23.3 

Ground/Air Task Oriented Radar (G/ATOR) 

1,641.37 

2,951.48 

3,110.54 

5.4 

89.5 

H-1 Upgrades (4BW/4BN) (H-1 Upgrades) 

4,060.64 

13,318.19 

13,538.48 

1.7 

233.4 

Infrared Search and Track (IRST) 

2,253.00 

2,240.41 

2,244.76 

0.2 

-0.4 

Integrated Defensive Electronic Countermeasures 
(IDECM) Block 4 

777.48 

1,218.17 

1,273.36 

4.5 

63.8 

Integrated Defensive Electronic Countermeasures 
(IDECM) Blocks 2/3 

1,660.27 

1,891.86 

1,257.00 

-33.6 

-24.3 

Joint Precision Approach and Landing System 

1,133.44 

1,919.50 

1,917.61 

-0.1 

69.2 

KC-130J Transport Aircraft (KC-130J) 

10,619.78 

9,943.88 

10,118.80 

1.8 

-4.7 

LHA 6 America Class Amphibious Assault Ship (LHA 6) 

3,560.70 

10,486.82 

10,674.30 

1.8 

199.8 

Littoral Combat Ship - Mission Modules (LCS Packages) 

7,336.98 

7,378.49 

6,418.17 

-13.0 

-12.5 

Littoral Combat Ship (LCS) 

2,512.72 

21,063.14 

21,281.43 

1.0 

746.9 
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Program name 

First full 
estimate of 
total 
acquisition 
cost 

December 

2016 

estimated 

total 

acquisition 

cost 

December 

2017 

estimated 

total 

acquisition 

cost 

Percentage 
change in 
total 
acquisition 
cost since 
December 
2016 

Percentage 
change in 
total 
acquisition 
cost since 
first full 
estimate 

LPD 17 San Antonio Class Amphibious Transport Dock 
(LPD17) 

13,113.42 

23,650.43 

23,744.07 

0.4 

81.1 

Mobile User Objective System (MUOS) 

7,526.23 

7,067.16 

7,075.03 

0.1 

-6.0 

MQ-4C Triton Unmanned Aircraft System (MQ-4C Triton) 

14,385.15 

15,697.35 

15,824.93 

0.8 

10.0 

MQ-8 Fire Scout 

2,928.35 

2,999.90 

3,103.84 

3.5 

6.0 

Multifunctional Information Distribution System (MIDS) 

1,460.21 

5,267.83 

5,661.54 

7.5 

287.7 

Navy Multiband Terminal (NMT) 

2,598.84 

2,468.86 

2,493.31 

1.0 

-4.1 

Next Generation Jammer Mid-Band 

7,843.32 

7,856.45 

8,133.25 

3.5 

3.7 

Offensive Anti-Surface Warfare Increment 1 (OASuW Inc 

i) 

1,577.57 

1,652.51 

1,885.17 

14.1 

19.5 

P-8A Poseidon Multi-Mission Maritime Aircraft (P-8A) 

34,750.79 

35,464.06 

36,589.99 

3.2 

5.3 

Ship to Shore Connector Amphibious Craft (SSC) 

4,426.20 

4,677.75 

4,752.17 

1.6 

7.4 

SSN 774 Virginia Class Submarine (SSN 774) 

67,684.81 

138,746.96 

139,807.62 

0.8 

106.6 

Standard Missile-6 (SM-6) 

6,382.87 

9,791.58 

8,610.05 

-12.1 

34.9 

Tactical Tomahawk RGM-109E/UGM 109E Missile 
(TACTOM) 

2,369.41 

7,607.94 

7,698.00 

1.2 

224.9 

Trident II (D-5) Sea-Launched Ballistic Missile UGM 

133A (Trident II Missile) 

52,863.96 

60,356.14 

60,718.46 

0.6 

14.9 

V-22 Osprey Joint Services Advanced Vertical Lift 

Aircraft (V-22) 

44,897.84 

65,856.98 

66,057.54 

0.3 

47.1 

VH-92A Presidential Helicopter Replacement Program 

4,999.14 

4,917.86 

4,842.17 

-1.5 

-3.1 

Joint DOD-wide 

Program name 

First full 
estimate of 
total 
acquisition 
cost 

December 

2016 

estimated 

total 

acquisition 

cost 

December 

2017 

estimated 

total 

acquisition 

cost 

Percentage 
change in 
total 
acquisition 
cost since 
December 
2016 

Percentage 
change in 
total 
acquisition 
cost since 
first full 
estimate 

Chemical Demilitarization-Assembled Chemical 

Weapons Alternatives (Chem Demil-ACWA) 

2,958.59 

13,709.55 

13,741.68 

0.2% 

364.5% 

F-35 Lightning II Program (F-35) 

239,291.99 

359,570.89 

360,097.25 

0.1% 

50.5% 


Source: GAO analysis of DOD data. | GAO-19-336SP. 

Note: We obtained data for this table from DOD’s Selected Acquisition Reports 
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DOD, and the Office of Management Budget 
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Figure 25: Percentage of Programs in Department of Defense’s (DOD) Last Six Portfolios, the Cost Growth of Which 
Remained Under Acceptable Limits over the Last Year, 5 Years, and Since First Full Estimate 

Percentage 
100 
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1 -1-year comparison (<2% growth)-1 


2013 2014 2015 2016 2017 2018 

I-5-year comparison (<10% growth)-1 


I | Percentage of programs that meet the metric 
| | Percentage of programs that do not meet the metric 

Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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Appendix VII: Program Rankings based on 
Total Acquisition Cost Change, Not Due to 
Quantity Changes, since First Full Estimates 


Tables 21 and 22 rank the programs in the 2018 portfolio by their total 
acquisition cost change across the managing military departments. We 
omitted amounts related to changes in quantity from the cost change 
calculations. Rankings are calculated based on the total cost, as well as 
the percentage change, in total acquisition expenditures since programs’ 
first full estimates. 


Table 21: Five Programs with the Lowest and Highest Total Acquisition Cost Changes by Total Dollars since First Full 
Estimates (fiscal year 2019 dollars in millions) 

Air Force 


Army 


Navy 


Program 

Change 
(in dollars) 

Program 

Change 
(in dollars) 

Program 

Change 
(in dollars) 

Lowest cost change 

1. KC-46A 

-7,851.91 

1.JLTV 

-2,761.52 

1. DDG51 

-25,066.81 

2. JDAM 

-3,346.45 

2. AMF 

-2,407.79 

2. MIDS 

-5,039.60 

3. HC/MC-130 

Recap 

-1,577.44 

3. AH-64E New Build 

-930.40 

3. CVN 78 

-2,213.62 

4. SDB II 

-745.75 

4. IAMD 

-660.01 

4. AIM-9X Blkll 

-871.69 

5. SBIRS High 

-470.33 

5. PAC-3 MSE 

-198.94 

5. AMDR 

-706.02 

Highest cost change 

1. EELV 

39,607.54 

1. UH-60M Black Hawk 

11,786.87 

1. V-22 

40,945.63 

2. AMRAAM 

13,427.50 

2. GMLRS/ 

GMLRS AW 

11,234.73 

2. SSN 774 

34,542.19 

3. MQ-9 Reaper 

3,202.24 

3. CH-47F 

10,101.76 

3. DDG 1000 

19,451.31 

4. GPS OCX 

2,570.00 

4. AH-64E 

Reman ufactu re 

5,967.27 

4. Trident II 

16,068.45 

5. JASSM-ER 

1,648.47 

5. MQ-1C Gray Eagle 

2,994.42 

5. LPD 17 

9,547.40 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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Appendix VII: Program Rankings based on 
Total Acquisition Cost Change, Not Due to 
Quantity Changes, since First Full Estimates 



Table 22: Five Programs with the Lowest and Highest Total Acquisition Cost Changes by Percentage since First Full 

Estimates 

Air Force 


Army 


Navy 


Program 

Change 

(percent) 

Program 

Change 

(percent) 

Program 

Change 

(percent) 

Lowest percentage change 

1. JDAM 

-87 

1. AH-64E New Build 

-35 

1. MIDS 

-345 

2. HC/MC-130 Recap 

-17 

2. AMF 

-26 

2. DDG51 

-147 

3. KC-46A 

-16 

3. IAMD 

-12 

3. AIM-9X Blkll 

-19 

4. B61 Mod 12 LEP 

TKA 

-15 

4. JLTV 

-11 

4. AMDR 

-11 

5. SDB II 

-14 

5. PAC-3 MSE 

-2 

5. IDECM Block 4 

-6 

Highest percentage change 

1. EELV 

204 

1. GMLRS/ 

GMLRS AW 

567 

1. H-1 Upgrades 

216 

2. AMRAAM 

110 

2. CH-47F 

280 

2. LCS 

204 

3. MQ-9 Reaper 

108 

3. MQ-1C Gray Eagle 

263 

3. G/ATOR 

111 

4. GPS OCX 

69 

4. UH-60M Black Hawk 

81 

4. V-22 

91 

5. JASSM-ER 

65 

5. AH-64E 

Remanufacture 

74 

5. CEC 

78 


Source: GAO analysis of Department of Defense data. | GAO-19-336SP 
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Appendix VIII: Technology Readiness Levels 


Table 23: Technology Readiness Levels and Descriptions 


Technology readiness level 

Description 

Hardware/software 

Demonstration 

environment 

1 . 

Basic principles observed 
and reported 

Lowest level of technology 
readiness. Scientific research 
begins to be translated into applied 
research and development. 
Examples might include paper 
studies of a technology’s basic 
properties. 

None (paper studies and 
analysis) 

None 

2. 

Technology concept and/or 
application formulated 

Invention begins. Once basic 
principles are observed, practical 
applications can be invented. The 
application is speculative and there 
is no proof or detailed analysis to 
support the assumption. Examples 
are still limited to paper studies. 

None (paper studies and 
analysis) 

None 

3. 

Analytical and experimental 
critical function and/or 
characteristic proof of 
concept 

Active research and development is 
initiated. This includes analytical 
studies and laboratory studies to 
physically validate analytical 
predictions of separate elements of 
the technology. Examples include 
components that are not yet 
integrated or representative. 

Analytical studies and 
demonstration of non-scale 
individual components (pieces of 
subsystem) 

Lab 

4. 

Component and/or 
breadboard validation in 
laboratory environment 

Basic technological components 
are integrated to establish that the 
pieces will work together. This is 
relatively “low fidelity” compared to 
the eventual system. Examples 
include integration of “ad hoc” 
hardware in a laboratory. 

Low-fidelity breadboard. 
Integration of nonscale 
components to show pieces will 
work together. Not fully 
functional or form or fit but 
representative of technically 
feasible approach suitable for 
flight articles. 

Lab 

5. 

Component and/or 
breadboard validation in 
relevant environment 

Fidelity of breadboard technology 
increases significantly. The basic 
technological components are 
integrated with reasonably realistic 
supporting elements so that the 
technology can be tested in a 
simulated environment. Examples 
include “high fidelity” laboratory 
integration of components. 

High-fidelity breadboard. 
Functionally equivalent but not 
necessarily form and/or fit (size 
weight, materials, etc.). Should 
be approaching appropriate 
scale. May include integration of 
several components with 
reasonably realistic support 
elements/subsystems to 
demonstrate functionality. 

Lab demonstrating 
functionality but not form 
and fit. May include flight 
demonstrating 
breadboard in surrogate 
aircraft. Technology 
ready for detailed design 
studies. 
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Technology readiness level Description Hardware/software Demonstration 

environment 

High-fidelity lab 
demonstration or 
limited/restricted flight 
demonstration for a 
relevant environment. 
Integration of technology 
is well defined. 

testing a prototype in a high fidelity 
laboratory environment or in 
simulated realistic environment. 


6. System/subsystem model or 
prototype demonstration in a 
relevant environment 


Representative model or prototype 
system, which is well beyond the 
breadboard tested for TRL 5, is 
tested in a relevant environment. 
Represents a major step up in a 
technology’s demonstrated 
readiness. Examples include 


Prototype. Should be very close 
to form, fit and function. Probably 
includes the integration of many 
new components and realistic 
supporting elements/subsystems 
if needed to demonstrate full 
functionality of the subsystem. 


7. System prototype Prototype near or at planned Prototype. Should be form, fit Flight demonstration in 

demonstration in an operational system. Represents a and function integrated with representative realistic 

operational environment major step up from TRL 6, requiring other key supporting environment such as 

the demonstration of an actual elements/subsystems to flying test bed or 

system prototype in a realistic demonstrate full functionality of demonstrator aircraft, 

environment, such as in an aircraft, subsystem. Technology is well 

vehicle, or space. Examples substantiated with test 

include testing the prototype in a data. 


test bed aircraft. 


8. Actual system completed and Technology has been proven to Flight-qualified hardware Developmental Test and 

“flight qualified” through test work in its final form and under Evaluation in the actual 

and demonstration expected conditions. In almost all system application. 

cases, this TRL represents the end 
of true system development. 

Examples include developmental 
test and evaluation of the system in 
its intended weapon system to 
determine if it meets design 


specifications. 


9. Actual system “flight proven” Actual application of the technology Actual system in final form Operational Test and 

through successful mission in its final form and under mission Evaluation in operational 

operations conditions, such as those mission conditions. 

encountered in operational test and 
evaluation. In almost all cases, this 
is the end of the last “bug fixing” 
aspects of true system 
development. Examples include 
using the system under operational 
mission conditions. 


Source: GAO and its analysis of National Aeronautics and Space Administration data. | GAO-19-336SP 
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Appendix IX: Comments from the 
Department of Defense 


OFFICE OF THE UNDER SECRETARY OF DEFENSE 

3000 DEFENSE PENTAGON 
WASHINGTON. DC 20301-3000 


ACQUISITION 
AND SUSTAINMENT 

Ms. Shelby Oakley 

Director. Contracting and National Security 
U.S. Government Accountability Office 
441 G Street. NW 
Washington. DC 20548 

Dear Ms. Oakley, 

This is the Department of Defense (DoD) response to the Government Accountability 
Office (GAO) Draft Report, GAO-19-336SP, “WEAPONS SYSTEM ASSESSMENT: Limited 
Use of Knowledge-based Practices Continues to Undercut DoD's Investments," dated April 9, 
2019 (GAO Code 102769). 

The Department remains committed to driving down the costs of our weapon systems and 
reducing the time it takes to deliver them to our warfighters. DoD continues to strive to 
implement knowledge-based acquisition practices in all of its Major Defense Acquisition 
Programs. In addition, DoD acknowledges the findings of awarding non-competitive contracts 
to a large number of programs, and DoD agrees that competition is the best way to reduce price, 
even as the American defense industrial base is evolving. The Department is committed to 
implementing and effectively executing initiatives to ensure the tax payer is getting the best 
return on investment. 

This Report documents the Department's focus on achieving our National Defense 
Strategy. Although there has been cost growth in some programs, the preponderance of that cost 
growth is a result of increased quantities of units since program inception. The decisions to 
increase the quantity of specific programs has ultimately resulted in lower per unit cost in many 
programs and is consistent with our National Defense Strategy. 

The Department appreciates the opportunity to comment on the Draft Report. My point 
of contact for this effort is Mr. Philip Rodgers at 703-692-5492. 


Sincerely. 



Stacy A. Cummings 

Principal Deputy Assistant Secretary of Defense 
(Acquisition Enablers) 



Page 211 


GAO-19-336SP Weapon Systems Annual Assessment 






Appendix X: GAO Contacts and Staff 
Acknowledgments 




GAO Contact 

Shelby S. Oakley, (202)512-4841 oroakleys@gao.gov 


Principal contributors to this report were Christopher R. Durbin, Assistant 
Director; Brenna Derritt, Program Assessments Analyst-in-Charge; 
Acknowledgments Marcus C. Ferguson, Portfolio Analysis Analyst-in-Charge; Dennis A. 

Antonio, Emily Bond; Jasmina Clyburn; Tana M. Davis; Wendy P. 

Smythe; and Robin M. Wilson. Other key contributors included Cheryl K. 
Andrew, Stephen Babb, David B. Best, Edwin B. Booth, Raj Chitikila, 
Matthew T. Crosby, Kevin Dooley, Arthur Gallegos, Jeffrey L. Hartnett, 
Rich Horiuchi, Justin M. Jaynes, Helena Johnson, J. Kristopher Keener, 
Jill N. Lacey, Travis J. Masters, LaTonya D. Miller, Diana Moldafsky, 

Scott Purdy, Carl Ramirez, Beth Reed Fritts, Ronald E. Schwenn, Charlie 
Shivers III, Jay Tallon, Brian A. Tittle, Bruce H. Thomas, Nathan A. 
Tranquilli, Abby C. Volk, J. Andrew Walker, Alyssa B. Weir, and Khristi A. 
Wilkins. 
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Table 24: GAO Staff Responsible for Individual Program Assessments 

Program Name 

Primary Staff 

Army Programs 

Armored Multi-Purpose Vehicle (AMPV) 

Charlie Shivers III, Eli DeVan 

CH-47F Modernized Cargo Helicopter (CH-47F Block II) 

William Allbritton, Cale Jones 
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Jessica Berkholtz , Wendy Smythe 
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Navy and Marine Corps Programs 
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Jeffrey L. Hartnett, Sara Rizik 

Amphibious Combat Vehicle Phase 1 Increment 1 (ACV 1.1) 

Matthew M. Shaffer, Holly Williams 

Amphibious Ship Program (LPD 17 Flight II) 

Holly Williams, Sarah Tempel 
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Program Name 

Primary Staff 

CH-53K King Stallion (CH-53K) 

Lauren Wright, Victoria Klepacz 

CVN 78 Gerald R. Ford Class Nuclear Aircraft Carrier / Advanced 
Arresting Gear 

(CVN 78/AAG) 

Jessica Karnis, Burns Eckert 

DDG 1000 Zumwalt Class Destroyer (DDG 1000) 

Laurier Fish, Ann Halbert-Brooks 

DDG 51 Arleigh Burke Class Guided Missile Destroyer - Flight III (DDG 
51 Flight III) 

Katherine M. Pfeiffer, Laura Jezewski 

Ground/Air Task Oriented Radar (G/ATOR) 

Joe E. Hunter, Claire Li 

Guided Missile Frigate (FFG(X)) 

Sean Merrill, Jillena Roberts 

Infrared Search and Track (IRST) 

Zachary J. Sivo, Jay Still 

Joint Precision Approach and Landing System (JPALS) 

Stephen V. Marchesani, Jennifer A. Dougherty 

Littoral Combat Ship Mission Modules (includes LCS) (LCS Packages) 

Tonya Woodbury, Chris Zakroff 
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Jillena Roberts, Christopher Lee 

MQ-4C Triton Unmanned Aircraft System (MQ-4C Triton) 

Erin Stockdale, James Kim 
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Heather Barker Miller, Jocelyn Yin 
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Teague Lyons, Jon Muchin 

SSBN 826 COLUMBIA Class Submarine (SSBN 826) 

Nathaniel Vaught, Laura Jezewski 

SSN 774 Virginia Class Submarine Block V (SSN 774 Block V) 

Jenny Shinn, James Madar 

T-AO 205 John Lewis Class Fleet Replenishment Oiler (T-AO 205 
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Matthew J. Ambrose, Jocelyn Yin 

VH-92A Presidential Helicopter (VH-92A) 

Bonita Oden, Anne McDonough 

Air Force Programs 

Advanced Pilot Training (APT) 

Jean Lee, Marvin Bonner 

B-2 Defensive Management System - Modernization (B-2 DMS-M) 

Megan Setser, Don Springman 

B-52 Radar Modernization Program (RMP) (B-52 RMP) 

Matthew C. Metz, Suzanne Sterling 

Combat Rescue Helicopter (CRH) 

Sean Seales, Matt Drerup 

F-15 Eagle Passive Active Warning Survivability System (F-15 
EPAWSS) 

Matthew Drerup, LeAnna Parkey 

Family of Advanced Beyond Line-of-Sight Terminals (FAB-T) 

Andrew Berglund, Bruna Oliveira 

Global Positioning System III (GPS III) 

Jonathan Mulcare, Patrick Breiding 

Global Positioning System III Follow-On Production (GPS INF) 

Jonathan Mulcare, Andrew Berglund 

KC-46A Tanker Modernization (KC-46A) 

Katheryn Hubbell, Nathaniel Vaught 

Military Global Positioning System (GPS) User Equipment Increment 1 
(MGUEIncI) 

Andrew Redd, Erin Carson 

Next Generation Operational Control System (OCX) 

Patrick Breiding, Juli Steinhouse 
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Program Name 

Primary Staff 

Small Diameter Bomb Increment II (SDB II) 

Suzanne Sterling, John Crawford 

Space Fence Ground-Based Radar System Increment 1 (Space Fence 
Inc 1) 

Laura Hook, Scott Purdy 

Utility Helicopter Replacement Program (UH-1N Replacement) 

Lindsey Cross, Gina Flacco 
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Joint Department of Defense Programs 

F-35 Lightning II Joint Strike Fighter (JSF) Program (F-35) 

Jen Baker, Desiree E. Cunningham 


Source: GAO. | GAO-19-336SP 
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